Tutorial - Cloud file server

Summary

This tutorial is targeted:

• Companies lacking possibilities to host file delivery solutions on-prem.

Wishing to:

• Use their existing cloud infrastructure to deliver files.

• Have transfers go at maximum speed, with high security - no account passwords communicated in emails or similar channels.

• Upload previews of work done, as H.264 (.mp4) for clients to review.

• Receive material from clients, and deliver final work - establish a complete file sharing solution.

• Make it easy for employees/project managers to manage the storage, share directories and monitor ongoing deliveries.

This tutorial is a complete walkthrough covering:

• Setting up a server running within a AWS EC2 instance.

• Creating an AWS S3 bucket for file storage.

• Connecting the bucket to the server and installing Accsyn.

• As an employee on-prem, transmit a couple of files to a new client for a certain project.

Note: In this tutorial we are relying on Amazon AWS for cloud infrastructure, the process is similar for setting up Accsyn in other cloud environments such as Google Cloud Platform och Microsoft Azure. 

Scenario

Creating the server

We assume you:

• Possess an existing Amazon account that is active and ready to use.

• Administrating from a Linux workstation. If you are at Windows, the only difference is how you configure your SSH client to use the AWS private key for accessing your EC2 instance.

1. Sign on to AWS.

2. If first time use, remember to switch to the region closest to you.

3. Go to EC2 Dashboard and choose [ LAUNCH INSTANCE ].

4. You can choose any server of type, that supports running Java 1.7 or later. In this tutorial we will go for a Ubuntu Server 18.04 LTS AMI.

5. Choose an instance type with at least 2 vCPUs and 1GB or RAM, then [ Review and Launch ].

6. Review and then click [ Launch ]. Remember to save the key pair if this is your first time and decide to your create a new,  we assume you have saved the private key in /root/.ssh/aws.pem.

7. To be enable Accsyn to server files, a range of network ports needs to be opened, go to Security Groups section and select the security group created with your EC2 instance.

8. Beneath Inbound tab, choose [ Edit ] and then [ Add rule ].

9. As the first rule, choose HTTPS (port 443, tcp) rule.

10. As the second rule, choose Custom TCP with range "45190-45209" with source "0.0.0.0/0, ::/0". (Each concurrent file transfer consumes one port, if you believe you will have a lot of clients transferring files simultaneously - add a much wider range as for example "45000-46000".

11.  Hit [ Save ] when you are done.

The server is now being launched, meanwhile we setup the S3 bucket.

Creating the storage

You can of course use an existing AWS S3 bucket, mount any preferred cloud storage or even increase the server local disk space to harbor your files. In this tutorial, we will create a dedicated S3 bucket for the purpose:

1. Go to Services>S3 section within your AWS.

2. Hit [ + Create Bucket ].

3. Choose a DNS compliant bucket name, in this tutorial we use "accsynstorage".

4. Double check region is the same as your server instance. Finish the wizard using default settings (public access off and so on).

To be able to access the S3 bucket, we need to create policy:

1. Go to My Security Credentials (IAM, located beneath your account name in upper right corner).

2. Go to Policies and click [ Create policy ].

3. Go to JSON tab and paste the json shown below.

4. Click [ Review policy ], give it a name, for example: "AmazonS3AccsynStorageFullAccess".

5. Finish up by clicking [ Create policy ].

We recommend creating a separate user for accessing the bucket:

1. Go to Users section within your IAM and click [ Add user ].

2. Choose a name, in this tutorial we will use "accsyn".

3. Choose Programmatic access as access type, hit [ Next: permissions ].

4. Choose Attach existing polices directly and locate the policy you created above (AmazonS3AccsynStorageFullAccess).

5. Click [ Next:tags ] and [ Next: review ], finish up by clicking [ Create user ].

6. Note the Access key ID and the Secret access key for later use. Note: The secret access key can only be viewed once, you will have to recreate the user if you forget to save it away.

If you wish to use an existing user:

1. Open user and choose [ Create access key]. 

2. Note the Access key ID and the Secret access key for later use. Note: The secret access key can only be viewed once, you will have to recreate the user if you forget to save it away.

Policy JSON:

{

    "Version": "2012-10-17",

    "Statement": [

        {

            "Effect": "Allow",

            "Action": [

                "s3:ListBucket"

            ],

            "Resource": [

                "arn:aws:s3:::accsynstorage"

            ]

        },

        {

            "Effect": "Allow",

            "Action": [

                "s3:PutObject",

                "s3:GetObject",

                "s3:DeleteObject"

            ],

            "Resource": [

                "arn:aws:s3:::accsynstorage/*"

            ]

        }

    ]

}

We are now ready to mount the S3 bucket at your EC2 instance.

Connecting storage to server

First we need to logon to your newly created EC2 instance for the first time:

1. Within your AWS console, go to Services>EC2 and Running instances.

2. Copy the IPv4 Public IP that AWS gave your server, for example "13.48.131.237".

3. Open a terminal and type the following: ssh -i /root/.ssh/aws.pem ubuntu@13.48.131.237.  If it complains about bad permissions, do: chmod 600 /root/.ssh/aws.pem.

4. Be root: sudo -s.

We will mount the S3 bucket with fuse and the s3fs filessystem, which needs to be installed:

1. First make sure Ubuntu is up to date: apt-get update -y.

2. Install fuse: sudo apt-get install automake autotools-dev fuse g++ git libcurl4-gnutls-dev libfuse-dev libssl-dev libxml2-dev make pkg-config -y. (On RHEL/CentOS: sudo yum install -y automake fuse fuse-devel gcc-c++ git libcurl-devel libxml2-devel make openssl-devel).

3. Install s3fs: 

git clone https://github.com/s3fs-fuse/s3fs-fuse.git

cd s3fs-fuse

./autogen.sh

./configure --prefix=/usr --with-openssl

make

sudo make install

Verify by running: which s3fs, it should show a path (/usr/bin/s3fs).

Finally, mount the S3 bucket:

1. Create file /etc/passwd-s3fs with permissions 640 and content based on the  Access key ID and the Secret access key for user "accsyn", for example:  AKIA1CCNI6YH6XLHIXPD:7dvU+IM6BgMqVRKJwDv/i4K/H9FP/D/U0EKPr8oi. 

2. Create the mount point: mkdir /accsynstorage.

3. Find out the service endpoint url for your AWS region, they are listed here: https://docs.aws.amazon.com/general/latest/gr/s3.html. In this tutorial it will be s3.eu-north-1.amazonaws.com.

4. Run: s3fs accsynstorage -o use_cache=/tmp,allow_other,uid=1000,mp_umask=002,multireq_max=5,url=https://s3.eu-north-1.amazonaws.com /accsynstorage. (Note: 1000 us the UID of ubuntu user)

5. Verify it by running: df -h /accsynstorage , you should see 256TB of free space!

(Optional) Have bucket mounted when directory is accessed (autofs, recommended as s3 buckets tends to dismount over time):

1. Install autofs: sudo apt-get install autofs

2. Add this line to /etc/auto.master: /- /etc/auto.s3bucket --timeout=30

3. Create /etc/auto.s3bucket with contents: /accsynstorage -fstype=fuse,use_cache=/tmp,allow_other,uid=1000,mp_umask=002,multireq_max=5,url=https://s3.eu-north-1.amazonaws.com  :s3fs\#accsynstorage

4. Reload autofs: sudo service autofs reload

(Optional) Make sure filesystem is mounted on boot:

1. Open up /etc/fstab in a text editor as root.

2. Add the following line: s3fs#accsynstorage /accsynstorage fuse _netdev,allow_other,umask=002,uid=1000,gid=1000,use_cache=/root/cache 0 0.

3. Save the file and test by rebooting the instance. When you log in, the filesystem should be available.

You are now ready to install Accsyn.

Install Accsyn

The following guide is a short summary of the installation process described in detail here: Accsyn Admin Manual

1. Register your domain @ https://customer.accsyn.com, in this tutorial we will register "mycorp".

2. Follow the guide to initialise domain and following the ling to your newly initialised domain ("https://mycorp.accsyn.com").

3. Install server; when the guide instructs you to install the Accsyn daemon, copy the link to the Linux installer and download in on the EC2 instance: wget -c https://mycorp.accsyn.com/app/accsyn-daemon-unix.sh. (Remainder: replace "mycorp" with the name of your Accsyn domain). Install Java as required by Accsyn: sudo apt-get install -y default-jdk. Run the installer: chmod 755 accsyn-daemon-unix.sh && sudo ./accsyn-daemon-unix.sh. Enter the 6-digit PIN number when asked, finish the installation and have service/daemon start.

4. Network; the guide will ask you to configure your firewall, you have already done that within your AWS security policies - simply mark port 443 and double check port range. Note: Accsyn daemon DOES NOT listen to any of these ports 24/7, it will only start a listening process during file transfer init - software firewalled to accept incoming connections from remote client WAN IP only.

5. Root share; Browse to were the disk is mounted on server [/accsynstorage], (optional) give root share a name. Note: Accsyn can only access files below this path, to able to access more directories you will have to add further root shares later.

6. Finish; Conclude setup by choosing a proper name for your Accsyn, leave it and the domain name will be used.

By now you have a fully working file transfer solution that can be used by external user to receive and send back large file packages at high speed (i.e. an accelerated encrypted FTP server replacement). We are now going to continue and show how to transmit a couple of files to a user.

Transmitting files to client

First you would want to invite your employee to Accsyn that is supposed to coordinate file transfers, in this tutorial emelie@mycorp.com. We will utilise this account below when transmitting files:

1. Logon to Accsyn and hit the [ Invite ] button at dashboard, or go to ADMIN>USERS and click [ + Invite user ].

2. Enter "emelie@mycorp.com" and click [ Next ].

3. Choose "Employee" clearance - giving them full access to the accsynstorage root share, invite users and send out files.

4. She will now get an invitation by Email with instructions on how to enter their personal Accsyn password.

Next, have employee download and install the Accsyn desktop app - required for communicating files with your Accsyn server in the cloud:

1. Logon to your Accsyn and go to GET APP section.

2. Download the installer for your platform and run it, consider our user manual if you are having troubles with installation: User Manual.

Lets now assume your employee have a movie file on her desktop called named A001_C011_09187Ia.mp4 that is to sent to client john@mail.com:

1. Launch Accsyn and login using account emelie@mycorp.com.

2. Drag-n-drop the movie onto Accsyn and choose Transmit option.

3. Choose to invite new user and enter "john@mail.com". Click [ Choose ] when you are done.

4. We want to organise files into project folders at our cloud server, cancel autostart by clicking [ No, I want to review ]. Note: If you let transfer submit here, files will land in the default Accsyn home share for user - default location will be folder accsyn/<Email> beneath root storage (accsynstorage).

5. Switch to browse mode by clicking [ Browse ] radio button.

6. Click on accsynstorage root share in list.

7. Create a new project folder, for example "myproj".

8. Enter the folder and finally click [ SUBMIT ] in the lower right corner. A dialog will have you confirm invite of user, add a welcome message here if you want.

Notes:

• The file will first be uploaded to the project folder within AWS S3 bucket from employees computer, after that the file will be delivered to user.

• User will get an invitation Email with instructions on how to download the the desktop app for accelerated and smooth transfers, they can of course also choose to open the transfer package and preview the movie in their web browser.

• User will have to enter their login and password to preview and download the files you send, this way you maintain integrity in case someone gets a hold of user's Email account or similar action.

For a complete guide on how to manage Accsyn on a day-to-day basis, check out our Manager Guide.

Further actions to take from here

• If you want user to be able to upload to the project folder @ AWS S3 bucket (root share), simply create a "share" within Accsyn and give user the appropriate permissions on directories. Then you can invite more users to collaborate @ this share.

• If you do not want home shares to be created, go to ADMIN>SETTINGS>Share and disable home shares. Also delete all home shares to clean up.

Thanks for reading this tutorial, contact support@accsyn.com if you have specific questions. For more resources and tutorials, please head over to https://support.accsyn.com.