Botomon

Synopsis

Use the Botomon utility to upload, download, or syncronize files between your local drive and a bucket on Amazon's S3 web service.

Installation

The Botomon gui and runner are available as standalone application executables, or as live python scripts. You have the option of installing either.

Executable

Windows

  • Create a desktop shortcut that points to [DROPBOX]\pipeline\pub\exe_win\botomon.exe
  • Double click on the desktop shortcut to launch Botomon
  • OSX

  • Create a desktop shortcut that points to [DROPBOX]/pipeline/pub/exe_osx/botomon
  • Double click on the desktop shortcut to launch Botomon
  • Full python environment

    Windows

    1. Navigate to [DROPBOX]/pipeline/pub/batch and double-click the "bootstrap.bat" file
    2. Answer "N" four times when the cmd shell asks if you'd like to overwrite your scheduled tasks.
    3. Create a desktop shotcut that points to [DROPBOX]/pipeline/pub/python/botomon.py
    4. Double click on the desktop shortcut to launch Botomon

    OSX

    Ask Raphe to install for now

    Settings

    Before you can transfer any data with Botomon, you need to fill out and save your settings preferences. Click on the "Settings" tab.

    Local root The path to the folder on your local hard drive that you'd like to sync. Click on the "..." button to choose a directory.
    Remote root The path to the folder on S3 that contains folders you'd like to sync. The depot sync root for modelers is "spaces/".
    Include prefixes A comma separated list of folders to include during synchronization. If no prefixes are specified, everything is sync'd. Modelers only have permission to access "work/" and "source/", therefore modelers should enter "work/,source/" in this field (otherwise uploads and downloads outside those folders will be triggered and then fail).
    Exclude prefixes A comma separated list of folders to exclude during synchronization. If a folder is entered in both the "Include prefixes" and "Exclude prefixes" fields, the folder will be excluded.
    Bucket name The name of the bucket on S3 to sync to. For modelers this bucket is named "lofteye".
    Access key The hexidecimal code generated by aws and distributed to you by Floored admins (probably Dustin or Judy) that defines your user and gives you permission to access certain files on Amazon's S3 web service.
    Secret key The hexidecimal code generated by aws and distributed to you by Floored admins (probably Dustin or Judy) that pairs with the Access key to enable Botomon to log you into a specified bucket on Amazon's S3 web service.

    Once you've entered your settings, click the "Save settings and refresh connection" button; every time you restart Botomon, it will connect to Amazon's S3 web service using these settings (i.e., you only have to enter them once). Botomon_runner will also use these preferences to connect to S3.

    Using the GUI

    Use the Botomon gui to add remote folders you'd like to download, work in, and synchronize to, then click the "download," "upload," or "sync" buttons to transfer data.

    ImportChoose File -> Import to import a .csv or .btmn project file
    Filter Use the filter to reduce the (potentially huge) number of remote folders to choose from.
    Refresh Use the two refresh buttons to refresh the Remote folders list and Local folders list, respectively. This is useful if you've added a remote or local folder since launching Botomon.
    Remote folders The list of folders on S3 inside the Remote root (defined in settings) that are available to be sync'd to.
    Local folders The list of local folders that you've added and that are available to sync.
    Browse Click this button to browse folders in the Local root (defined in settings). If a folder is selected in Local folders, that folder will be opened.
    InfoSelect a folder and click this button to print it's remapping details in the console
    -> Click this button to add a remote folder to the list of local folders you'd like to sync.
    Verbose Click this button to access more verbose output in the console (log output is always verbose).
    Preview Click this button to preview files that will be uploaded, downloaded, or deleted when any of the sync buttons are clicked (no files will be transferred).
    Download Click this button to safely download files from the folder(s) selected in the Local folders list (if none are selected, all folders will be sync'd.
    Upload Click this button to safely upload files from the folder(s) selected in the Local folders list (if none are selected, all folders will be sync'd.
    Sync Click this button to safely upload and download files from the folder(s) selected in the Local folders list (if none are selected, all folders will be sync'd).
    Progress bar Displays the progress of completed tasks. Included tasks are "upload", "download", and "delete."
    Open logs folder Click this button to browse the Botomon Gui and Runner logs
    View log Click this button to view the log for the current Botomon session.
    HelpChoose Help -> Web manual to view open Botomon documentation in a web browser
    ExitChoose File -> Exit to quit the Botomon application

    Importing a project to work on

    The lead modeler will create and distribute .btmn (or .cvs) project files for other modelers to import by choosing File -> Import.

    Working directly on a space

    It is possible to work directly on a space without a project file. This should only be done with permission from the lead modeler. Choose a space from the list of remote folders and click the '->' button.

    Resolving conflicts

    If two users have modified the same file, conflicts will be generated before a file transfer; no files will be transferred and the user will be prompted to resolve the conflicts manually. This can be done by simply deleting or renaming the offending files and clicking the relevant transfer button again.

    Recovering files

    It is (hopefully) always possible to recover files that have been accidentally removed from S3 or your local drive.

    Recovering files from S3

    If files are accidentally removed from S3, do not fear. Every version of every file can be recovered. Ask a Floored admin to recover files from the version history; provide them with the filename and date the file was removed.

    Recovering files from your local drive

    All files that are removed by Botomon from local projects are sent to either the Recycling Bin (Windows) or the Trash (OSX). Locate the file that was accidentally trashed, right click on it and choose "Restore" (Windows) or "Put back" (OSX)

    Removing a project

    If you're finished working on a project and don't want to sync it anymore, simply delete it from your local machine. Don't worry -- it won't be deleted from the server. For example, if I had these projects listed in

    ... and I wanted to delete the "54-59 project," I would navigate to my local root directory (in my case "C:\Users\Rizzo the Rat\Desktop\botomon\lofteye") and move the "54-59_Broadway" folder to the Recycling Bin or the Trash.

    Using the Runner

    Windows

    To use the Botomon runner in a Windows cmd shell, type:

    C:\Python27\python.exe [DROPBOX]\pipeline\pub\python\botomon_runner.py -h
    

    OSX

    To use the Botomon runner in a OSX terminal, type:

    python [DROPBOX]/pipeline/pub/python/botomon_runner.py -h
    

    Usage

    If the -h flag is present, usage information will be displayed that looks something like this:

    C:\Users\Rizzo the Rat\Dropbox (Floored)\pipeline\dev\exe_win>botomon_runner.exe
    usage: botomon_runner.exe [-h] [-sync] [-upload] [-download] [-preview] [-log]
                              [-verbose] [-cleanup] [-local_root LOCAL_ROOT]
                              [-s3_bucket_name S3_BUCKET_NAME]
                              [-s3_access_key S3_ACCESS_KEY]
                              [-s3_secret_key S3_SECRET_KEY]
    
    This script uses information in the Botomon gui prefs file to safely transfer
    data. Values can be overriden with optional arguments.
    
    optional arguments:
      -h, --help            show this help message and exit
      -sync                 if set, this script will upload and download files
      -upload               if set, this script will upload files
      -download             if set, this script will download files
      -preview              if set, no files will be transfered
      -log                  print the path to the log and exit
      -verbose              if set, console output will be verbose (log is always
                            verbose)
      -cleanup              if set, all multi-part uploads that have failed on the
                            bucket will be canceled
      -local_root LOCAL_ROOT
                            path to the local folder
      -s3_bucket_name S3_BUCKET_NAME
                            the name of the target bucket on your AWS S3 account
      -s3_access_key S3_ACCESS_KEY
                            the hexadecimal access key used to access the
                            specified S3 bucket
      -s3_secret_key S3_SECRET_KEY
                            the hexadecimal secret key used to access the
                            specified S3 bucket
    

    Using the runner on a timer

    Windows

    If you installed the full python environment for Windows, there is already a scheduled task in the Windows Task Scheduler -- open the Windows Task Scheduler (C:\ProgramData\Microsoft\Windows\Start Menu\Programs\Administrative Tools\Task Scheduler) and click on "Botomon sync at midnight" to view and/or edit it's details. It is recommended that you turn on "Wake the computer to run this task."

    If you're using the executable, create a new "Basic task" and give it the path to the executable with the optional -sync flag, like this:

    OSX

    Tool / Doc Information

    Prefered method?: Yes
    
    Tool authors: raphe
    Tool state: Wip
    
    Doc authors: raphe
    Doc state: Wip
    Doc created on: October 22, 2014, 8:58 am
    Doc last edited by: raphe
    Doc last edited on: October 28, 2014, 11:38 am