accsyn transfer job specification
Source and destination party
accsyn is a p2p file transfer protocol which means that each transfer job can only have one unique source and one unique destination.
A source or destination can be:
The organization; on-prem servers at main site (hq), running the accsyn daemon app.
A user; Identified by the Email, running the accsyn desktop app or daemon app.
A site; Identified by a unique name, running the accsyn daemon app.
Web browser; For downloads and uploads using web browser.
For the organization party, source or destination files can reside on multiple (root) shares within the same job:
Example: Transfer one file from share1 and another from share2, that can reside on different servers.
Job count limit
Although now limits on the amount of jobs are enforced by accsyn, when amount reaches 500+ a substantial degrade of performance occurs and UI will become immeasurable.
The recommended approach is to reduce the amount of jobs and instead have multiple tasks within each job.
Giving an example were the API in an automatic workflow submits a sync job for each file updated and everyday, if thousands of files are updated accsyn will soon reach its limits and the job listing in desktop app /web app will be overflowed.
A better approach would be to let the API load a daily sync job for each source-destination pair (create if not exists) and add tasks to that job instead. This would reduce the amount of jobs drastically, and will make job listing much more readable when it comes to find other important jobs that otherwise would totally drown.
Large jobs with a lot of files
accsyn transfers files using the same algorithm as *NIX rsync which means that a list of files with size and modification dates are sent to receiving end in order to determine which files need to be sent.
For very large file transfers containing a lot of smaller files in deep lengthy folder structures, accsyn might run out of RAM during file transfer init and in those cases it is recommended to split a job in multiple tasks.
For example when doing project backup with accsyn, instead of sending the entire root share or directory - send each project directory as individual tasks and set "task_bucketsize" setting to "1":
Job JSON specification
When is the job JSON used?
accsyn jobs are internally submitted in JSON format, even from the desktop app.
When submitting a job through the API our using a file with CLI, the correct job JSON must be provided as specified throughout the rest of this document.
Structure of job JSON
The general structure of job JSON:
Job attributes; (optional) Typically name of job (attribute name: "code"). The name of job is auto generated using filename of first task if not given.
Tasks: List of tasks - files and directories to transfer, see party and path notation below.
Settings; (optional) Additional job settings, see below for a complete listing.
Metadata; (optional) Additional job settings.
Note: tasks can be omitted if just a single file, and be replaced with job attributes "source" and "destination".
accsyn party and path notation
The accsyn job JSON requires "tasks" entry containing on ore more source & destination definitions, accsyn have adopted the rsync notation on how to define a target and a path:
Where party can be:
<domain>; Main premises/hq servers, for example domain "myorg" with accsyn URL https://myorg.accsyn.com.
<username>; A user, for example "firstname.lastname@example.org".
<site=id|code>; A remote site server, for example "stockholm".
And path can be:
C:\Users\john\Desktop\image.png; A local path at user end.
/Volumes/projects/reference.tif; An absolute path on a root share.
share=projects/reference.tif ; Same notation, but references the share "projects" and leaving accsyn to complete path with configured prefix for server platform. This is also called the accsyn path notation.
myproject/reference.jpg ; Assume file paths being relative default root share, equivalent to share=projects/myproject/reference.jpg.
racing2019_grade/test.abc; Used in conjunction with user as target, deliver file into relative folder "racing2019_grade" at user end.
Destination paths can be left out if other party is a site or an employee, this is called "path mirroring", and is suitable for keeping servers and/or workstations in sync when it comes to file structure.
If party is omitted, accsyn interprets this as the domain party - file is to be sent to or from hq.
Some job JSON examples
Send a file to a user
Deliver the file "/projects/lfm/114/290/lfm_114_290.jpg" (requires a share to be configured /projects) to user "email@example.com", storing at relative path "projects/lfm/114/290/". Short format, single task:
Send two files to multiple users
Send out a small preview, that will be written to "preview" subfolder at the user download location, and a large movie written to "movie" subfolder, destined two different users that will be invited if not exists:
Upload to home share
Upload folders "/Users/john/Desktop/delivery" & "quicktimes" to user firstname.lastname@example.org's home share, have job reside in queue "low_prio", with task identifiers:
Download from user share
Download file "20180413/script_v001.pdf" from user share to local folder "C:\Users\John\Download\accsyn":
Leaving out the trailing slash will copy the file to file "accsyn" on the receiving side.
Tasks can be supplied as a list, leaving to accsyn to apply task identifiers.
Upload to share
Upload folder "/Users/john/Desktop/delivery" to project share "thefilm" into subfolder "from_john/20180413":
Push to site
Sync folder "thefilm/010" on share "projects" to site "stockholm01", paths mirrored (default when site are involved):
Upload from site
Download rendered folder "projects/got/sc01/sh01/render/got_sc01_sh01_comp_v012" from site "google" to site "stockholm":
Upload folder "racing2019_grade/davinci_files", at share "projects" from site "stockholm" back to hq, with metadata (can be picked up by post process hooks):
Skip existing files
Upload file "final_export.mov", at share "projects" from user to share "racing2019_grade", but not overwriting it if exists and size or modification date differ:
Upload a large folder, excluding all files ending with "tmp" and is only numbers:
Note: multiple exclude statements are separated by an escaped comma - \, . This means that an escaped comma cannot be used in exclude expressions.
accsyn supports sending a subset of a numbered file sequence, this is handy when you do not want to send an entire directory with a huge amount of files but instead want to do a selection.
Transfer a part of a file sequence from one site to another, mirrored paths (requires VPN direct connection or both server setup with NAT port forwarding of accsyn protocol ports):
The prefix "share=" can be omitted if the party is not a user, i.e. sending to/from the organization hq/site.
Source/destination can be specified without tasks, for single file/directory transfers. Tasks can be a list instead of a JSON, each transfer task while then be numbered by the order they appear in list.
The following scenarios can also be submitted directly from the commandline, for example: "accsyn job create myorg:/projects/lfm/114/290/lfm_114_290.jpg email@example.com:projects/lfm/114/290/lfm_114_290.jpg".
If destination exists and is a directory, accsyn behaves as unix "rsync" - the file ends up beneath the folder.
If file exists were accsyn were supposed to save a folder, transfer will fail.
The following job settings can be provided on submit: