rclone/notes.txt

Todo
  * FIXME: More -dry-run checks for object transfer
  * FIXME ls is very slow also - need to run mtimes in parallel
  * Check logging in various parts
  * Make logging controllable with flags (mostly done)
  * progress meter would be nice! Do this by wrapping the Reader with a progress bar
  * Do bandwidth limit by wrapping the Reader too
      * Maybe using https://jra-go.googlecode.com/hg/linkio/ which will work for multiple
        uploads or downloads. 
      * code.google.com/p/mxk/go1/flowcontrol - only does one flow at once
      * Or maybe put into swift library.
  * -timeout: Make all timeouts be settable with command line parameters
  * Check the locking in swift module!
  * Windows paths? Do we need to translate / and \?
  * Make a fs.Errorf and count errors and log them at a different level
  * add -modify-window flag - fs should keep knowledge of resolution

Ideas
  * optimise remote copy container to another container using remote
    copy if local is same as remote
  * Allow subpaths container:/sub/path
  * stats
  * look at auth from env in s3 module - add to swift?

Make a wrapper in connection which
  * measures bandwidth and reports it
  * limits bandwidth using Reader and Writer
  * for a pool of all individual connectinos
  * does timeouts by setting a limit, seeing whether io has happened
    and resetting it if it has

Need to make directory objects otherwise can't upload an empty directory
  * Or could upload empty directories only?
  * Can't purge a local filesystem because it leaves the directories behind

s3
--

Can maybe set last modified?

https://forums.aws.amazon.com/message.jspa?messageID=214062

Otherwise can set metadata

Returns etag and last modified in bucket list
Detect Fs from path to make operations consistent everywhere 2012-12-29 12:35:41 +01:00			`Todo`
Implement check and help commands 2012-12-31 18:31:19 +01:00			`* FIXME: More -dry-run checks for object transfer`
Make ModTime fall back to LastModified header Which means it no longer needs to report an error which simplifies the code 2013-01-02 16:21:55 +01:00			`* FIXME ls is very slow also - need to run mtimes in parallel`
Detect Fs from path to make operations consistent everywhere 2012-12-29 12:35:41 +01:00			`* Check logging in various parts`
Implement sync, -dry-run and fix logging * Implement sync command * Implement String() interface for Fs * Sort out logging of FsObject~s * Implement -dry-run, -verbose and -quiet 2012-12-31 17:40:34 +01:00			`* Make logging controllable with flags (mostly done)`
Detect Fs from path to make operations consistent everywhere 2012-12-29 12:35:41 +01:00			`* progress meter would be nice! Do this by wrapping the Reader with a progress bar`
			`* Do bandwidth limit by wrapping the Reader too`
			`* Maybe using https://jra-go.googlecode.com/hg/linkio/ which will work for multiple`
			`uploads or downloads.`
			`* code.google.com/p/mxk/go1/flowcontrol - only does one flow at once`
			`* Or maybe put into swift library.`
Implement sync, -dry-run and fix logging * Implement sync command * Implement String() interface for Fs * Sort out logging of FsObject~s * Implement -dry-run, -verbose and -quiet 2012-12-31 17:40:34 +01:00			`* -timeout: Make all timeouts be settable with command line parameters`
Detect Fs from path to make operations consistent everywhere 2012-12-29 12:35:41 +01:00			`* Check the locking in swift module!`
			`* Windows paths? Do we need to translate / and \?`
			`* Make a fs.Errorf and count errors and log them at a different level`
Implement sync, -dry-run and fix logging * Implement sync command * Implement String() interface for Fs * Sort out logging of FsObject~s * Implement -dry-run, -verbose and -quiet 2012-12-31 17:40:34 +01:00			`* add -modify-window flag - fs should keep knowledge of resolution`
Detect Fs from path to make operations consistent everywhere 2012-12-29 12:35:41 +01:00
Update notes 2012-12-23 18:46:03 +01:00			`Ideas`
Detect Fs from path to make operations consistent everywhere 2012-12-29 12:35:41 +01:00			`* optimise remote copy container to another container using remote`
			`copy if local is same as remote`
Update notes 2012-12-23 18:46:03 +01:00			`* Allow subpaths container:/sub/path`
			`* stats`
Detect Fs from path to make operations consistent everywhere 2012-12-29 12:35:41 +01:00			`* look at auth from env in s3 module - add to swift?`
Make checking and uploading run in parallel using channels Runs much faster and uses much less memory (thanks to walking the filesystem into a channel too). 2012-11-29 23:13:58 +01:00
			`Make a wrapper in connection which`
			`* measures bandwidth and reports it`
			`* limits bandwidth using Reader and Writer`
			`* for a pool of all individual connectinos`
			`* does timeouts by setting a limit, seeing whether io has happened`
			`and resetting it if it has`

Fix downloads - missing is the modified check so it downloads everything 2012-12-01 09:18:58 +01:00			`Need to make directory objects otherwise can't upload an empty directory`
			`* Or could upload empty directories only?`
Detect Fs from path to make operations consistent everywhere 2012-12-29 12:35:41 +01:00			`* Can't purge a local filesystem because it leaves the directories behind`

			`s3`
			`--`

			`Can maybe set last modified?`

			`https://forums.aws.amazon.com/message.jspa?messageID=214062`
Make download work properly skipping not changed files Reorganise FsObject and factor more stuff into methods Note that os.Chtimes() isn't setting the same precision (us) as we are reading from os.Lstat() (ns) so the modification times don't roundtrip properly yet. 2012-12-01 11:53:48 +01:00
Detect Fs from path to make operations consistent everywhere 2012-12-29 12:35:41 +01:00			`Otherwise can set metadata`
Make download work properly skipping not changed files Reorganise FsObject and factor more stuff into methods Note that os.Chtimes() isn't setting the same precision (us) as we are reading from os.Lstat() (ns) so the modification times don't roundtrip properly yet. 2012-12-01 11:53:48 +01:00
Detect Fs from path to make operations consistent everywhere 2012-12-29 12:35:41 +01:00			`Returns etag and last modified in bucket list`