code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawnZEanlyzay_QlEAL0CWpyZcRTyN7vay8U"
 nickname="Carlo"
 subject="Best thing since sliced bread"
 date="2012-11-21T13:08:57Z"
 content="""
It's amazing how perfect git annex is for my use case.

I record music, which results in a bunch of big files that don't change, and a bunch of small files which change often.

I also need these files to be preserved for the long haul, and be verified now and then. They also need to be backed up three or four times, because they're original files! Keeping track of all that was a headache before git annex, and I was also constantly awake at night because it was impossible to verify if everything's still okay.

Now I just dump everything into my archive annex and I know where everything is, and that it's safe, period. I even set myself up a raspberry pi with a usb drive adapter and put some spindown code in a cron job. Now I have a hard drive at rest that I can access from anywhere at any time. The whole thing gets backed up by Amazon Glacier, but if Amazon starts getting pesky I can just add another backend and sync it up. I can even version my small files if I want to by putting them directly into git. This entire thing would have been such a huge undertaking but now it's easy. I dump everything into the annex and to the actual backups whenever I get round to it. It just has a natural flow to it.

Et voila. Professional grade backups, at home, independant from any specific vendor. I absolutely never have to worry about my files going anywhere, and if I get super paranoid, I can just \"git annex fsck\" and mathematically prove to myself everything's still there.

It's totally sad how many creative people will be losing their life's work in a couple of years because they rely on regular filesystems on their USB drives. Stuff like DropBox and SpiderOak is better than nothing but trusting single vendor to not screw up or mess with your life's work is just plain creepy.

I just have a huge, warm fuzzy feeling now. If this sounds like I'm relentlessly hyping git annex, it's because I am! It deserves it!

"""]]


code:
This is work in progress, since there is now a [[special_remotes/hook]] for users to plug in whatever they want as a remote, here's my recipe for using tahoe-lafs as a remote, this is a copy and paste the relavent section from my .git/config file

        tahoe-store-hook = tahoe put $ANNEX_FILE tahoe:$ANNEX_KEY
        tahoe-retrieve-hook = tahoe get tahoe:$ANNEX_KEY $ANNEX_FILE
        tahoe-remove-hook = tahoe rm tahoe:$ANNEX_KEY
        tahoe-checkpresent-hook = tahoe ls tahoe:$ANNEX_KEY 2>&1 || echo FAIL

Where `tahoe:` is a tahoe-lafs alias, ideally you should create a new alias (DIR-CAP or whatever the terminolgy is) to store your files, I just used the default `tahoe:` alias for testing.

The only quirk I've noticed is this...

<pre>
$ git annex whereis .
whereis frink.jar (2 copies)
  	084603a8-7243-11e0-b1f5-83102bcd7953  -- here (testtest)
   	1d1bc312-7243-11e0-a9ce-5f10c0ce9b0a
ok
</pre>

1d1bc312-7243-11e0-a9ce-5f10c0ce9b0a is my [[!google tahoe-lafs]] remote, but there is no label/description on it. The checkpresent-hook was a little confusing when I was setting it up, I'm currently unsure if I am doing the right thing or not with my hook. My get and put commands are a little verbose for now, i might redirect it to /dev/null once I am happier with the overall performance/behaviour my setup.

Other than the quirks above, I am able to put and get files from my tahoe-lafs remote. The only thing that I have not figured out is how to "remove a file" on the remote to free up space on the remote.

repo_name: kakafarm/git

license: unknown


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawmBUR4O9mofxVbpb8JV9mEbVfIYv670uJo"
 nickname="Justin"
 subject="whereis labels"
 date="2011-04-29T13:08:35Z"
 content="""
You should be able to fix the missing label by editing .git-annex/uuid.log and adding

    1d1bc312-7243-11e0-a9ce-5f10c0ce9b0a tahoe
"""]]


code:
[[!comment format=mdwn
 username="http://joey.kitenet.net/"
 nickname="joey"
 subject="comment 2"
 date="2011-04-29T15:24:56Z"
 content="""
If `tahoe ls` outputs only the key, on its own line, and exits nonzero if it's not present, then I think you did the right thing.

To remove a file, use `git annex move file --from tahoe` and then you can drop it locally.
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawkSq2FDpK2n66QRUxtqqdbyDuwgbQmUWus"
 nickname="Jimmy"
 subject="comment 3"
 date="2011-04-29T15:33:24Z"
 content="""
@justin, I discovered that \"git annex describe\" did what I wanted

@joey, yep that is the behaviour of \"tahoe ls\", thanks for the tip on removing the file from the remote.

It seems to be working okay for now, the only concern is that on the remote everything is dumped into the same directory, but I can live with that, since I want to track biggish blobs and not lots of small little files.
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawkSq2FDpK2n66QRUxtqqdbyDuwgbQmUWus"
 nickname="Jimmy"
 subject="comment 4"
 date="2011-04-29T16:17:11Z"
 content="""
I've just tried to use the ANNEX_HASH_ variables, example of my configuration

    git config annex.tahoe-store-hook 'tahoe mkdir $ANNEX_HASH_1 && tahoe put $ANNEX_FILE tahoe:$ANNEX_HASH_1/$ANNEX_KEY'
    git config annex.tahoe-retrieve-hook 'tahoe get tahoe:$ANNEX_HASH_1/$ANNEX_KEY $ANNEX_FILE'
    git config annex.tahoe-remove-hook 'tahoe rm tahoe:$ANNEX_HASH_1/$ANNEX_KEY'
    git config annex.tahoe-checkpresent-hook 'tahoe ls tahoe:$ANNEX_HASH_1/$ANNEX_KEY 2>&1 || echo FAIL'
    git annex initremote library type=hook hooktype=tahoe encryption=none
    git annex describe 1d1bc312-7243-11e0-a9ce-5f10c0ce9b0a library

It's seems to work quite well for me now, I did run across this when I tried to drop a file locally, leaving the file on my remote

jtang@x00:/tmp/annex3 $ git annex drop .
drop frink.sh (checking library...) (unsafe)
  Could only verify the existence of 0 out of 1 necessary copies
  Try making some of these repositories available:
  	1d1bc312-7243-11e0-a9ce-5f10c0ce9b0a  -- library
  (Use --force to override this check, or adjust annex.numcopies.)
failed
drop t/frink.jar (checking library...) (unsafe)
  Could only verify the existence of 0 out of 1 necessary copies
  Try making some of these repositories available:
  	1d1bc312-7243-11e0-a9ce-5f10c0ce9b0a  -- library
  (Use --force to override this check, or adjust annex.numcopies.)
failed
git-annex: 2 failed
1|jtang@x00:/tmp/annex3 $

I do know that the files exist in my library as I have just inserted them, it seemed to work when I didnt have the hashing, it appears that the checkpresent doesn't seem to pass the ANNEX_HASH_* variables (from the limited debugging I did)
"""]]


code:
[[!comment format=mdwn
 username="http://joey.kitenet.net/"
 nickname="joey"
 subject="comment 5"
 date="2011-04-29T18:01:04Z"
 content="""
I've corrected the missing `ANNEX_HASH_*` oversight. (It also affected removal, btw.)
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawkSq2FDpK2n66QRUxtqqdbyDuwgbQmUWus"
 nickname="Jimmy"
 subject="comment 6"
 date="2011-04-29T20:11:08Z"
 content="""
Cool, that seems to make things work as expected, here's an updated recipe

<pre>
git config annex.tahoe-store-hook 'tahoe mkdir tahoe:$ANNEX_HASH_1/$ANNEX_HASH_2 && tahoe put $ANNEX_FILE tahoe:$ANNEX_HASH_1/$ANNEX_HASH_2/$ANNEX_KEY'
git config annex.tahoe-retrieve-hook 'tahoe get tahoe:$ANNEX_HASH_1/$ANNEX_HASH_2/$ANNEX_KEY $ANNEX_FILE'
git config annex.tahoe-remove-hook 'tahoe rm tahoe:$ANNEX_HASH_1/$ANNEX_HASH_2/$ANNEX_KEY'
git config annex.tahoe-checkpresent-hook 'tahoe ls tahoe:$ANNEX_HASH_1/$ANNEX_HASH_2/$ANNEX_KEY 2>&1 || echo FAIL'
git annex initremote library type=hook hooktype=tahoe encryption=none
git annex describe 1d1bc312-7243-11e0-a9ce-5f10c0ce9b0a library
</pre>

I just needs some of the output redirected to /dev/null.

(I updated this comment to fix a bug. --[[Joey]])
"""]]


code:
[[!comment format=mdwn
 username="zooko"
 ip="97.118.97.117"
 subject="request for information, plus some ideas"
 date="2011-05-14T05:07:17Z"
 content="""
Hey Jimmy: how's this working for you now? I would expect it to go slower and slower since Tahoe-LAFS has an O(N) algorithm for reading or updating directories.

Of course, if it is still fast enough for your uses then that's okay. :-)

(We're working on optimizations of this for future releases of Tahoe-LAFS.)

I'd like to understand the desired behavior of store-hook and retrieve-hook better, in order to see if there is a more efficient way to use Tahoe-LAFS for this.

Off to look for docs.

Regards,

Zooko
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawkSq2FDpK2n66QRUxtqqdbyDuwgbQmUWus"
 nickname="Jimmy"
 subject="comment 8"
 date="2011-05-14T10:02:26Z"
 content="""
@joey thanks for the update in the previous comment, I had forgotten about updating it.

@zooko it's working okay for me right now, since I'm only putting fairly big blogs on stuff on to it and only things that I *really* care about. On the performance side, if it ran faster then it would be nicer :)
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawmsz4weoPXV2oEtv3zpo9dOxn_SEPz-7Iw"
 nickname="Zooko"
 subject="more notes about Tahoe-LAFS performance"
 date="2014-04-21T07:18:06Z"
 content="""
In case anyone is reading this thread: https://github.com/zooko/tahoe-lafs/blob/3c13c138cf09e83d2f8001888e2a7de85564d406/docs/frontends/key-value-store.rst
"""]]


code:
Hi,

Is anyone having difficulties with the assistant and gpg-agent?  Particularly with multiple repos syncing to a crypt backup?

It seems to work for a while and then according to the error logs I can't decrypt the manifest...I'm wondering if there is some sort of locking involved that only lets one repo sync to a gcrypt repo at a time?  I can't think why the gpg-agent would stop providing the key.

I also have a rather horrible time with gpg-agent in general:  I now start each repos git annex assistant individually in the shell (from which I can confirm the gpg-agent is working) rather than launch the web-app direct.


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawnXybLxkPMYpP3yw4b_I6IdC3cKTD-xEdU"
 nickname="Matt"
 subject="comment 1"
 date="2014-04-29T10:16:11Z"
 content="""
hmmm - a command line based git annex sync always works.  So either the assistant has a separate locking mechanism or the assistant is somehow losing the gpg-agent environment.  Does the assistant restart/respawn itself and lose access to the gpg-agent env?

I'm somewhat clutching at straws here.
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawnXybLxkPMYpP3yw4b_I6IdC3cKTD-xEdU"
 nickname="Matt"
 subject="comment 2"
 date="2014-04-30T12:25:55Z"
 content="""
I see in the comments here a possible explanation...

https://git-annex.branchable.com/bugs/GPG_passphrase_repeated_prompt/
"""]]


code:
When issuing a `git annex info`, I get:

~~~
$ git-annex info
repository mode: indirect
trusted repositories: fatal: Not a git repository: '/home/micas/Music/.git' fatal: Not a git repository: '/home/micas/Music/.git' 0
semitrusted repositories: 3
	00000000-0000-0000-0000-000000000001 -- web
 	85f8a5ea-6278-11e2-9978-ebb59e8f37a2 -- here (Music annex backup)
 	9aff38f2-6447-11e2-8c89-ef50e6c0ea6c -- backupone (Music annex backupone)
untrusted repositories: 0
transfers in progress: none
available local disk space: 174.28 gigabytes (+1 megabyte reserved)
local annex keys: 5348
local annex size: 25.62 gigabytes
annexed files in working tree: 5374
size of annexed files in working tree: 25.68 gigabytes
bloom filter size: 16 mebibytes (1.1% full)
backend usage:
	SHA256: 10692
	SHA256E: 30
~~~

The troubling part (I think) is `trusted repositories: fatal: Not a git repository: '/home/micas/Music/.git' fatal: Not a git repository: '/home/micas/Music/.git' 0`

Is there a command I can use to show all the uuid of known remotes? I thought I had marked all remotes pointing to /home/micas/Music/.git as `dead`

Is there another reason for the output? How can I get trusted repos back to 0?

**EDIT:**

I removed the folder (was not a git repo at the time of the error) `/home/micas/Music` and the error went away.


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="209.250.56.227"
 subject="comment 1"
 date="2013-12-29T20:10:14Z"
 content="""
Yeah, git-annex will skip a repository if its directory doesn't exist, but if the directory exists, it expects it to be a git repository, and will run git commands that use it (to get its config specifically).

The ugly message is doing just what it should; letting you know you have a problem.
"""]]


code:
I just briefly tried git-annex. It's an interesting concept.

My main frustration so far: I find it difficult to visualise and use the annex. Shell script contortions can obviously show you anything, but I don't have the ready knowledge to compose them. It would be nice to have some more user-friendly commands or even a GUI.

'annex whereis' is fine for one file, but unweildly for many (e.g. a directory whose contents are always together, never divided between repositories). Is it possible to treat a given directory and all its contents as a single object?

It is not clear what files are immediately available. Sometimes you don't care about annexed files that aren't stored in the current repository. Might it be nice to temporarily remove or hide symlinks for files that are not here right now? Then you could treat the repository more like a normal file heirarchy. Or how about something like 'git annex ls' to show only currently available files?

assistant and sharebox fs sound like great basic synch options, but it would be really cool to have a fuse fs somewhere between raw git-annex and assistant. Putting a file in the directory would trigger git-annex to add it and sync. rm <file> on that new file would not work by default because it's the only copy. rm --annex to get rid of the file from all repos. mv/cp <file> <remote> to transfer between repos. ls to show only the files that are here now, ls --annex to display complete information in a clear way (well-formatted, colorized). Something like that.

This has just been a random brain dump from a new user, hopefully it made some sense.


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="4.154.0.140"
 subject="comment 1"
 date="2013-07-20T19:51:19Z"
 content="""
It's pretty easy to configure ls to show broken symlinks in a different color. Run dircolors, and alias ls='ls --color==auto'. Then you can see which files are not there at a glance.

The Linux file manager's I've tried (nautilus and thunar) also shows which files are present; you get a X on the icon of annexed files that are not present.
"""]]


code:
[[!comment format=mdwn
 username="http://mylesenglish.myopenid.com/"
 ip="217.39.94.195"
 subject="comment 2"
 date="2013-09-16T19:21:31Z"
 content="""
This shell function behaves more like ls and shows the files that are present:

    ga-ls () {
      CWD=`pwd`
      cd ${1:-.} && git annex find | cut -d / -f 1 | uniq
      cd ${CWD}
    }

"""]]


code:
what is the work flow to get a file that is in git-annex out of there and into git? (current situation: `git-annex add`ed a bunch of pictures, later found make files in there which i'd rather have in git for proper source code control)

the most intuitive thing to do is `git unannex`, which at first seemed to do the right thing, but when committing there came the hook and everything was back to where it was before.

i could disable the hook as a workaround, but that doesn't smell like a good work flow.

the [[man page|git-annex]] does warn that `unannex` is only supposed to be used against unintentional `git annex add`s (probably meaning that it should be used before something is committed), but the alternatives it suggests (`git rm` and `git annex drop`) don't to what i want to do.

am i missing something or is there really no work flow for this? --[[chrysn]]


code:
[[!comment format=mdwn
 username="http://joey.kitenet.net/"
 nickname="joey"
 subject="comment 1"
 date="2011-02-02T00:39:10Z"
 content="""
Git-annex's commit hook does not prevent unannex being used. The file you unannex will not be checked into git anymore and will be a regular file again, not a git-annex symlink.

For example, here's a transcript:

<pre>
joey@gnu:~/tmp>mkdir demo
joey@gnu:~/tmp>cd demo
joey@gnu:~/tmp/demo>git init
Initialized empty Git repository in /home/joey/tmp/demo/.git/
joey@gnu:~/tmp/demo>git annex init demo
init demo ok
joey@gnu:~/tmp/demo>echo hi > file
joey@gnu:~/tmp/demo>git annex add file
add file ok
(Recording state in git...)
joey@gnu:~/tmp/demo>git commit -m add
[master 64cf267] add
 2 files changed, 2 insertions(+), 0 deletions(-)
 create mode 100644 .git-annex/WORM:1296607093:3:file.log
 create mode 120000 file
joey@gnu:~/tmp/demo>git annex unannex file
unannex file ok
(Recording state in git...)
joey@gnu:~/tmp/demo>ls -l file
-rw-r--r-- 1 joey joey 3 Feb  1 20:38 file
joey@gnu:~/tmp/demo>git commit
[master 78a09cc] unannex
 2 files changed, 1 insertions(+), 2 deletions(-)
 delete mode 120000 file
joey@gnu:~/tmp/demo>ls -l file
-rw-r--r-- 1 joey joey 3 Feb  1 20:38 file
joey@gnu:~/tmp/demo>git status
# On branch master
# Untracked files:
#   (use \"git add <file>...\" to include in what will be committed)
#
#	file
nothing added to commit but untracked files present (use \"git add\" to track)
</pre>
"""]]


code:
[[!comment format=mdwn
 username="http://joey.kitenet.net/"
 nickname="joey"
 subject="comment 2"
 date="2011-02-02T00:41:24Z"
 content="""
And following on to my transcript, you can then add the file to git in the regular git way, and it works fine:

<pre>
joey@gnu:~/tmp/demo>git add file
joey@gnu:~/tmp/demo>git commit
[master 225ffc0] added as regular git file, not in annex
 1 files changed, 1 insertions(+), 0 deletions(-)
 create mode 100644 file
joey@gnu:~/tmp/demo>ls -l file
-rw-r--r-- 1 joey joey 3 Feb  1 20:38 file
joey@gnu:~/tmp/demo>git log file
commit 225ffc048f5af7c0466b3b1fe549a6d5e9a9e9fe
Author: Joey Hess <joey@kitenet.net>
Date:   Tue Feb 1 20:43:13 2011 -0400

    added as regular git file, not in annex

commit 78a09cc791b875c3b859ca9401e5b6472bf19d08
Author: Joey Hess <joey@kitenet.net>
Date:   Tue Feb 1 20:38:30 2011 -0400

    unannex

commit 64cf267734adae05c020d9fd4d5a7ff7c64390db
Author: Joey Hess <joey@kitenet.net>
Date:   Tue Feb 1 20:38:18 2011 -0400

    add
</pre>
"""]]


code:
[[!comment format=mdwn
 username="http://joey.kitenet.net/"
 nickname="joey"
 subject="comment 3"
 date="2011-02-02T00:46:00Z"
 content="""
Sorry for all the followups, but I see now that if you unannex, then add the file to git normally, and commit, the hook *does* misbehave.

This seems to be a bug. git-annex's hook thinks that you have used git annex unlock (or \"git annex edit\") on the file and are now committing a changed version, and the right thing to do there is to add the new content to the annex and update the symlink accordingly. I'll track this bug over at [[bugs/unannex_vs_unlock_hook_confusion]].

So, committing after unannex, and before checking the file into git in the
usual way, is a workaround. But only if you do a "git commit" to commit
staged changes.

Anyway, this confusing point is fixed in git now!
"""]]


code:
Hi,

when running git annex add in my direct mode repository, since a few days ago I only get:

$ git annex add
git-annex: unknown response from git cat-file (":./Archiv/Someone missing",:./Archiv/Someone Like You Cover-fCvjvEGkTu4.flv)

The :./Archiv/Someone missing part strikes me odd because it so much looks like broken shell meta-character escaping in git-annex, but I doubt that because it stopped working just suddenly.


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="2001:4978:f:21a::2"
 subject="comment 1"
 date="2013-08-07T16:34:18Z"
 content="""
This is a bug in the 1.8.4 pre-release version of git. It will be fixed in the 1.8.4 release or another pre-release. git-annex version 4.20130802 has a workaround for this problem.
"""]]


code:
Several times now I've done something like:

    $ git annex unlock movie.avi
    $ mv /tmp/fixed.avi movie.avi
    $ git annex lock movie.avi

Oops, I just lost my fixed.avi!  That really feels like the right sequence of operations to me, so I'm always surprised when I make that mistake.  I would like to see the current `lock` renamed to something like `undo-unlock`, or have the behavior changed to be the same as `add`, or maybe warn and require a `--force` when the file has been changed.

If changing current behavior is undesirable, maybe `unlock` could just print a reminder that `git annex add` is the correct next step after making changes?

Failing that, I suppose I could slowly start to learn from my mistakes.


code:
[[!comment format=mdwn
 username="http://joey.kitenet.net/"
 nickname="joey"
 subject="comment 1"
 date="2012-01-07T17:15:31Z"
 content="""
Well, lock could check for modifications and require --force to lose them. But the check could be expensive for large files.

But `git annex lock` is just a convenient way to run `git checkout`. And running `git checkout` or `git reset --hard` will lose your uncommitted file the same way obviously.

Perhaps the best fix would be to get rid of `lock` entirely, and let the user use the underlying git commands same as they would to drop modifications to other files. It would then also make sense to remove `unlock`, leaving only `edit`.
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawnRai_qFYPVvEgC6i1nlM1bh-C__jbhqS0"
 nickname="Matthew"
 subject="comment 2"
 date="2013-09-07T12:26:46Z"
 content="""
I too was totally flummoxed by this.
"""]]


code:
[[!comment format=mdwn
 username="http://olivier.berger.myopenid.com/"
 nickname="obergix"
 subject="It seems we've been heard"
 date="2013-12-07T15:13:10Z"
 content="""
https://github.com/joeyh/git-annex/commit/cde099fe80cd2bc6345496c9266c41baf46c1976 : lock: Require --force.
"""]]


code:
I have no idea how to search for this here, so I'll just go the "lazy web" approach and just ask.

Say I have two "conference" repos. One is the famous [conference procedings](https://github.com/RichiH/conference_proceedings) repo, and another one is a totally unrelated repo of local conferences that are not of world-wide significance. Let's call this second repo `presentations`.

I would like to have my videos of both repos in a single repo.

Can I add the `conference procedings` repo as a git remote to the `presentations` repo and have it do the right thing?

In fact, I'm not even sure what the right thing would be here, I guess that's the first thing I would like to clear up. But I would like to do things like what the new [[metadata]] system does. For example, I would have only the "Debian" directory from `conference procedings` in my `presentations` repo.

How would that work? Would I need to do some [subtree merging](http://git-scm.com/book/ch6-7.html) magic? or `git subtree`? or submodules? or should i just use myrepos and pretend I never brought up this idea?

thanks! -- [[anarcat]]

related: [[tips/migrating_two_seperate_disconnected_directories_to_git_annex/]] - but that creates a merged repo...


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="209.250.56.244"
 subject="comment 1"
 date="2014-04-02T19:51:06Z"
 content="""
It might help if you think about these two different repositories as branches. You have 2 branches with different files in them, and you want to produce a third branch with some mix of the two.

I think git is perfectly capable of doing that. Where it gets hairy is dealing with merges when either of the 2 repositories change going forward. The same as if you've forked and modified source code, you will need to do *something* to resolve merges.
"""]]


code:
If I have an archive with a lot of big files, is it possible to get one of them without using the command line, and without it popping up on other clients of that repository?
Is it possible to have a folder that will not be synced with other clients but will still download the file when i copy it from archive?
.gitignore probably wont work because that will not get the file from archive.


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="209.250.56.87"
 subject="comment 1"
 date="2013-12-12T20:09:48Z"
 content="""
Sorry I took so long to answer. I think that it took me a few tries to understand your question.

You want to have a special folder on a single clone of a repository, and be able to move files to that folder from the archive folder, and have it get the file content from the archive repository. But you don't want that file content to spread to any of the other repositories.

This is possible to do by writing a custom [[preferred_content]] expression, and configuring the repository to use it. For simplicity, let's make the folder be called `archive/local`. This way, all the clients that do not use this preferred content expression will see that the file is still under the archive folder, and not want its content.

Now we can take the preferred content expression used by client repositories normally:

  ((exclude=*/archive/* and exclude=archive/*) or (not (copies=archive:1 or copies=smallarchive:1))) or (not copies=semitrusted+:1)

And modify it:

  ((exclude=*/archive/* and exclude=archive/* and (not */archive/local/*) and (not archive/local/*)) or (not (copies=archive:1 or copies=smallarchive:1))) or (not copies=semitrusted+:1)

You can use `git annex vifg` to edit the preferred content expressions, and paste that in. Should work. Untested.
"""]]


code:
I tried to update git-annex via cabal. It fails due to lens not installing:

    juh@sokrates:~$ cabal update
    Downloading the latest package list from hackage.haskell.org
    juh@sokrates:~$ cabal install git-annex
    Resolving dependencies...
    In order, the following will be installed:
    lens-4.2 (new version)
    DAV-0.6.2 (reinstall) changes: http-client-0.3.1.1 -> 0.3.3.1,
    http-client-tls-0.2.1.1 added, http-types-0.8.4 -> 0.8.5, lens-4.1.2 ->     4.2,
    network-2.4.1.2 added, optparse-applicative-0.7.0.2 added, xml-    conduit-1.2.0.1
    -> 1.2.0.2, xml-hamlet-0.4.0.8 added
    git-annex-5.20140707 (new package)
    Warning: Note that reinstalls are always dangerous. Continuing anyway...
    [1 of 1] Compiling Main             ( /tmp/lens-4.2-3107/lens-4.2/Setup.    lhs, /tmp/lens-4.2-3107/lens-4.2/dist/setup/Main.o )
    Linking /tmp/lens-4.2-3107/lens-4.2/dist/setup/setup ...
    Configuring lens-4.2...
    Building lens-4.2...
    Preprocessing library lens-4.2...

    src/Control/Lens/Internal/Zoom.hs:47:8:
        Could not find module `Control.Monad.Trans.Except'
        Perhaps you meant
          Control.Monad.Trans.Cont (from transformers-0.3.0.0)
          Control.Monad.Trans.Error (from transformers-0.3.0.0)
          Control.Monad.Trans.List (from transformers-0.3.0.0)
        Use -v to see a list of the files searched for.
    Failed to install lens-4.2
    cabal: Error: some packages failed to install:
    DAV-0.6.2 depends on lens-4.2 which failed to install.
    git-annex-5.20140707 depends on lens-4.2 which failed to install.
    lens-4.2 failed during the building phase. The exception was:
    ExitFailure 1

I am not an experienced user of cabal. So any hints to solve the conflicts are appreciated.


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="209.250.56.55"
 subject="comment 1"
 date="2014-07-09T17:58:48Z"
 content="""
It seems like you must have transformers-compat 0.1.1, and lens's dependency is slightly wrong, accepting that extremly old version. You'll need to upgrade it.
"""]]


code:
Is it possible to let git-annex check and update the "number of copies" or "whereis" information without recalculating the checksome?

The use case is this:

I have a very large "repo1": 300.000 files 1.5 TB.

I copy it using low level tools for performance (clone the partition, netcat over network, or whatever...)

Then i do "git annex init 'repo2'".
All files exist in both repos and the simlinks are valid, but git-annex does not know about it.
The "git-annex wehereis" only shows "repo1". How can I tell him without rehashing 1.6TB?

Thx


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="4.154.6.49"
 subject="comment 1"
 date="2012-12-01T18:40:46Z"
 content="""
Well, first of all, after copying a repository like this, you need to edit its .git/config and delete the annex.uuid setting. Otherwise, you will have two repositories with the same UUID, which is not good.

Once you've done that, run `git annex fsck` in the new repository and it will do what you want.
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawlgyVag95OnpvSzQofjyX0WjW__MOMKsl0"
 nickname="Sehr"
 subject="comment 2"
 date="2012-12-01T19:27:59Z"
 content="""
Ok, that did the trick, except that it recalculates all cehcksums, which is exactly wat I do not want, as it is unnecessary and takes at 40MB/s just too long. Any other way? I think about adding the whereis info by hand, which simply feels wrong!
"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="4.154.6.49"
 subject="comment 3"
 date="2012-12-01T19:37:26Z"
 content="""
git annex fsck --fast will skip the checksumming.
"""]]


code:
i want to relate a usability story that happens fairly regularly when I show git-annex to people. the story goes like this.

----

Antoine sat down at his computer saying, "i have this great movie collection I want to share with you, my friend, because the fair use provisions allow for that, and I use this great git-annex tool that allows me to sync my movie collection between different places". His friend Charlie, a Linux user only vaguely familiar with the internals of how his operating system or legal system actually works, reads this as "yay free movies" and wholeheartedly agrees to lend himself to the experiment.

Antoine creates a user account for Charlie on his home computer, because he doesn't want to have to do everything himself. "That way you can choose which movies you want, because you probably don't want my complete movie collection!" Charlie emphatically responds: "right, I only have my laptop and this USB key here, so I don't think I can get it all".

Charlie logs into Antoine's computer, named `marcos`. Antoine shows Charlie where the movies are located (`/srv/video`) through the file browser (Thunar, for the record). Charlie inserts his USB key into `marcos` and a new icon for the USB key shows up. Then Charlie finds a video he likes, copies and pastes it into the USB key. But instead of a familiar progress bar, Charlie is prompted with a dialog that says "Le système de fichiers ne gère pas les liens symboliques." (Antoine is french, so excuse him, this weird message says that the filesystem doesn't support symbolic links.) Puzzled, Charlie tries to copy the file to his home directory instead. This works better, but the file has a little arrow on it, which seems odd to Charlie. He then asks Antoine for advice.

Antoine then has no solution but to convert the git-annex repository into direct mode, something which takes a significant amount of time and is actually [[designated as "untrusted"|direct_mode]] in the documentation. In fact, so much so that he actually did [[screw up his repository magnificently|bugs/direct_command_leaves_repository_inconsistent_if_interrupted]] because he freaked out when `git-annex direct` started and interrupted it because he tought it would take too long.

----

Now I understand it is not necessarily `git-annex`'s responsability if Thunar (or Nautilus, for that matter), doesn't know how to properly deal with symlinks (hint: just dereference the damn thing already). Maybe I should file a bug about this against thunar? I also understand that symlinks are useful to ensure the security of the data hosted in `git-annex`, and that I could have used direct mode in the first place. But I like to track changes in git to those files, and direct mode makes that really difficult.

I didn't file this as a bug because I want to start the conversation, but maybe it should qualify as a usability bug. As things stand, this is one of the biggest hurdle in teaching people about git annex.

(The other being "how do i actually use git annex to sync those files instead of just copying them by hand", but that's for another story!)

-- [[anarcat]]


code:
Dear all,
           From some time now I am wondering about a way to index a set of files, lets say PDF documents. The idea is to have a unique identifier for each file and to cross-reference using this identifier. For instance, I use a project management (PM) software (web based) on a public server of my university. Then I have a set of tasks saying, review document X, or Y. And those documents are stored on an internal server of my lab.
I see several options:

1. Upload the required documents to the PM site and directly link
2. Share online my internal server and use the URL of the docs in the PM
3. Just use the unique identifier in the PM, and then look in git annex for that ID
4. Use some sort of document management system (DMS)

Options 1 and 2 are impractical for several reasons. Option 4 usually requires that your files are inside the DMS.
So my questions are:

* Do you think this is doable with git-annex?
* Is there an easy way to ask it: give me the document with this index?
* I think the best answer for this question is: git annex find --include '*' --format='${key} ${file}' | grep <KEY>
* And conversely, how do I find the key of a certain document?

Thanks in advance.
Best,
       Juan


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="209.250.56.55"
 subject="comment 1"
 date="2014-07-03T19:10:53Z"
 content="""
To make git-annex output the key of a file, run: `git annex lookupkey $file`

I don't know if the git-annex key is appropriate for your use-case. If the files never get changed, then it's a nice stable identifier. If ongoing changes are made to a file, and you want to link to the most recent version, the key would not be useful.

You might also look at git-annex's [[metadata]]; you could make up some metadata field and value and attach it to a file, and it would persist as the file was modified.
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawmTNrhkVQ26GBLaLD5-zNuEiR8syTj4mI8"
 nickname="Juan"
 subject="comment 2"
 date="2014-07-09T15:32:49Z"
 content="""
If I get this right, the key will change every time the file is changed.
If that's the case, seems that it won't be useful for my case.
Thanks for the time.
"""]]


code:
Hi, a new user trying out git-annex 3.20130124 here. So far looks really promising, thanks.

I setup a local annex using the assistant webapp. I then created a rsync cloud repo over ssh to my shell box in a nearby data center for it. Works ok. I already have ssh key setup inclding ssh-agent for quick logins to the server. It would be great if you could configure the assistant to use these existing keys instead of creating a new set of keys. Maybe keep the defaults as is, but provide the config for this hidden behind "trust me, I know what I'm doing" check box or something.

Thanks again, I keep researching.


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="4.154.3.125"
 subject="comment 1"
 date="2013-02-07T18:11:30Z"
 content="""
Hmm. The assistant already tries to cater to this case of having an existing key. When setting up a rsync or ssh remote, it first tries to ssh to the server without allowing a password to be prompted for. If this succeeds, it assumes you have a passwordless key and uses it. Otherwise, it sshes in once with a password prompt, and sets up its own key.
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawnwz07HJdp3nzLwCAl7OGKn_3m0tTO4Mxw"
 nickname="Teemu"
 subject="use existing ssh keys?"
 date="2013-03-02T12:05:30Z"
 content="""
I also found the key setup confusing. I have many identity files for several server / service combinations, so the webapp key setup for an ssh+rsync cloud server always failed with too many identification attempts before I had the sense to move my keys temporarily away from ~/.ssh

It would be really nice if the webapp would allow one either to pick an existing identity, or to create a new one with the aid of a couple of password logins.

Also, having the created key to be restricted to only running rsync in the server would improve the chances of 'Share with a friend' use case being used. I don't think many people will want to give out full shell access to their servers in order to share some files.
"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 nickname="joey"
 subject="comment 3"
 date="2013-03-03T16:57:22Z"
 content="""
When pairing with a local computer, both systems set up locked down ssh keys that only allow the pair to run git-annex on the repository being paired with. This does *not* involve full shell access.

The \"remote server\" necessarily involves you already having shell access to the remote server. However, the ssh key that the assistant generates is again locked down to only being able to run git-annex on a single repository on the remote server.
"""]]


code:
I would like to use git-annex to synchronize 2 directories in the same manner as unison.

I'm starting with 2 directories. There is an overlap of the same set of files in each directory, but each directory also has additional files as well.

I create a git annex in each directory but when I do a git pull it merges and produces conflicts on those files that are the same.

What is the correct workflow for this type of scenario?


code:
[[!comment format=mdwn
 username="http://www.joachim-breitner.de/"
 nickname="nomeata"
 subject="comment 1"
 date="2011-12-18T13:57:33Z"
 content="""
Are the files identical or different? I today did something like that with similar, but not identical directories containing media files, and git happily merged them. but there, same files had same content.

Also, make sure you use the same backend. In my case, one of the machines runs Debian stable, so I use the WORM backend, not the SHA backend.
"""]]


code:
[[!comment format=mdwn
 username="http://joey.kitenet.net/"
 nickname="joey"
 subject="comment 2"
 date="2011-12-19T18:24:59Z"
 content="""
I'd recommend using the SHA backend for this, the WORM backend would produce conflicts if the files' modification times changed.

[[syncing_non-git_trees_with_git-annex]] describes one way to do it.
"""]]


code:
after upgrading to git-annex 3, i'm stuck with diverging git-annex branches -- i didn't manage to follow this line in the directions:

> After this upgrade, you should make sure you include the git-annex branch when git pushing and pulling.

could you explain how to do that in a littel more detail? git pull seems to only merge master, although i have these ``.git/config`` settings:

    [branch "git-annex"]
    	remote = origin
    	merge = git-annex


code:
[[!comment format=mdwn
 username="http://joey.kitenet.net/"
 nickname="joey"
 subject="comment 1"
 date="2011-08-17T01:33:08Z"
 content="""
It's ok that `git pull` does not merge the git-annex branch. You can merge it with `git annex merge`, or it will be done
automatically when you use other git-annex commands.

If you use `git pull` and `git push` without any options, the defaults will make git pull and push the git-annex branch automatically.

But if you're in the habit of doing `git push origin master`, that won't cause the git-annex branch to be pushed (use `git push origin git-annex` to manually push it then). Similarly, `git pull origin master` won't pull it. And also, the `remote.origin.fetch` setting in `.git/config` can be modified in ways that make `git pull` not automatically pull the git-annex branch. So those are the things to avoid after upgrade to v3, basically.
"""]]


code:
Hi Joey,

it would be nice when views could take numeric comparisons as filters.

    git annex metadata -s length=273.0 john_cage_4_33.mp3

    git annex view length<=300

... here is the catch, < and > don't work well in shell, this needs some other Syntax. I think the underlying machinery (using numeric comparisons instead globs) should be quite trivial. Any Ideas about a Syntax?


code:
[[!comment format=mdwn
 username="cehteh"
 ip="217.8.62.137"
 subject="consistent syntax"
 date="2014-03-25T15:25:43Z"
 content="""
Further thinking led to the idea to use the test(1) like syntax to filter matches.

    git annex view length=.* -lt 300 -gt 30 -o -eq 273 bpm=.* -eq 0

"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="209.250.56.41"
 subject="comment 2"
 date="2014-03-26T17:53:28Z"
 content="""
I'm a little bit worried about the potential to reinvent SQL, badly. ;)

As shown in your example, once you have ranges, it's natural to also want disjunctions, and then probably parenthesized expressions, and suddenly things are very complicated.

Also, it's important that views remain reversable, so that committing a moved file in a view can unambiguously calculate the new metadata for it. I think that quickly becomes hard when adding these complications.
"""]]


code:
[[!comment format=mdwn
 username="cehteh"
 ip="217.8.62.137"
 subject="comment 3"
 date="2014-03-26T18:55:41Z"
 content="""
I'm a little bit worried about the potential to reinvent SQL, badly. ;)

As shown in your example, once you have ranges, it's natural to also want disjunctions, and then probably parenthesized expressions, and suddenly things are very complicated.

Also, it's important that views remain reversable, so that committing a moved file in a view can unambiguously calculate the new metadata for it. I think that quickly becomes hard when adding these complications.

"""]]


code:
[[!comment format=mdwn
 username="cehteh"
 ip="217.8.62.137"
 subject="comment 4"
 date="2014-03-26T20:03:51Z"
 content="""
I agree with you that things must stay simple. All what should be done is having the same effects like normal globs but adding arithmetic comparisons to it (could you think about a globbing extension over numeric values?). Then the generated views will have the same properties/semnatic as the normal glob'ed views without other side effects (if you want to go that far, this would even hold true for disjunct, parenthesized and otherwise complex expression).

Example (how it should work, except my bug report 'set metadata on wrong files')

    git annex metadata a.txt -s foo=bar -s num=1
    git annex metadata b.txt -s foo=baz -s num=2
    git annex metadata c.txt -s foo=barf -s num=3

    git annex view foo=bar* num=* -ne 2

should give
    ./bar/1/a.txt ./barf/3/c.txt

am I right now than one could

    mkdir -p ./baz/2
    mv /bar/1/a.txt ./baz/2

to change the metadata of a.txt, despite the foo=baz and num=2 fields where initially filtered out when creating the view?
If this assumption is true then having arithmetic filters, no matter how complex they are won't change the existing semantics over what globs do.

"""]]


code:
Hi

Is it possible to create a view which also includes files with no tag?

I use something like `git annex view 'rating=*'` to view files sorted by rating but this view does not include files which don‘t have a rating yet. What I was looking for is a way to show tagged files and untagged files in one view.


code:
[[!comment format=mdwn
 username="Xyem"
 ip="87.194.19.134"
 subject="comment 1"
 date="2014-03-25T08:54:23Z"
 content="""
This is a TODO:

http://git-annex.branchable.com/design/metadata/

    unmatched files in filtered branches

    TODO Files not matching the view should be able to be included in the filtered branch, in a special location, an \"other\" directory.

In the meantime, I do this before switching to the view:

    git annex metadata -s fieldIwant?=untagged

This shows the files without any \"fieldIwant\" in the directory \"untagged\". Afterwards, I could (but don't need to in my workflow, it automatically gets removed), do this to remove it:

    git annex metadata -s fieldIwant-=untagged

What the \"?=\" does is add that metadata only if the \"fieldIwant\" hasn't been set at all.
"""]]


code:
[[!comment format=mdwn
 username="http://ypid.wordpress.com/"
 ip="213.153.84.215"
 subject="Perfect"
 date="2014-03-25T20:11:26Z"
 content="""
This works great. Nice workaround, thanks very much.
"""]]


code:
Hello everyone,
I would like to know if this is normal behavior or if it's a problem with my repository:

Whenever I set a view with

git annex view attr="\*"'

a new branch representing the selected view gets created, as expected. The problem is that when I switch back to master ('git checkout master' or even 'git annex vpop') the view branch stays there, and all subsequent operations on the annex also consider the view branch, resulting a great slowdown if one has done many views (attr="this", attr="that", etc.). Is this normal? If so, why is it necessary for the branch to stay on? Does it speed up going back to the branch? Redoing git annex view attr="*" does not seem to take less time.

Am I doing it wrong? Should I be deleting used view branches on my own? How?

thanks for your replies.

**EDIT:** I just found out that even if I delete view branches with git branch -D "views/attr=_" (which I'm not sure I should be doing), the branches are still checked when doing "git annex unused". That is, "git annex unused" lists "checking..." a whole lot of past views/branches which are not even there anymore (not listed with "git branch"). I also suspect that this is preventing deleted (git-rm) files from being collected from "unused". Is this a problem with my repo? Any way to fix this?

=== git-annex version: **5.20140529-gb71f9bf** ===


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="108.236.230.124"
 subject="comment 1"
 date="2014-06-04T18:04:30Z"
 content="""
You can delete the view branches if you like. I have considered making moving views clean up old branches, but I have also considered reusing existing view branches when popping back to an old view.

git-annex unused can't look at branches that aren't there. Perhaps you have pushed the view branches to a remote repository, and it's checking those branches of the remote? See git branch -a.

I think that it makes sense for unused to ignore (local) view branches, since these are by definition supposed to be views of an existing branch, so looking at the branch should be sufficient (and if the view is out of date and has files that have since been deleted from the branch, the user's intent is not to preserve those from unused reaping). So, made that change.
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawmYMS6tQ8q9k1fylGyA9Q6fdFzRBotME2Q"
 nickname="tom"
 subject="EDIT reply comment:"
 date="2014-06-04T19:44:18Z"
 content="""
> *You can delete the view branches if you like. I have considered making moving views clean up old branches, but I have also considered reusing existing view branches when popping back to an old view.*

Alright, I'd very much welcome anyone of the two possibilities as I'm making heavy use of views and would benefit from any of the two solutions.

> *Perhaps you have pushed the view branches to a remote repository, and it's checking those branches of the remote? See git branch -a.*

That's it! Solution for future reference: The remote branches referenced in local repo did not exist anymore in any repo (I had deleted them earlier from the remote) so \"git push origin --delete \<branchname\>\" wouldn't work. This other command worked very well in cleaning up the overwhelming missing remote views all in one go: \"git fetch --prune \<remote\>\"

> *I think that it makes sense for unused to ignore (local) view branches, since these are by definition supposed to be views of an existing branch, so looking at the branch should be sufficient. So, made that change.*

Thanks a lot! It makes perfect sense, looking forward to it on the next release.

Now, it seems that even after pruning the view branches on the remote, *unused* would still not collect deleted files. I'm wondering if this is because there are other offline repos (backups which are not practical to take out of their resting place) I have not yet synchronized. I can see their main branches with git branch -a: remotes/oldbackup/git-annex, remotes/oldbackup/master, remotes/oldbackup/synced/git-annex, remotes/oldbackup/synced/master. Does this mean that until all repos (even the offline ones I never sync and work mostly as a backup for old stuff) are synchronized, 'unused' won't collect the deleted files? If so, is there a workaround for this?

"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawmYMS6tQ8q9k1fylGyA9Q6fdFzRBotME2Q"
 nickname="tom"
 subject="added comment"
 date="2014-06-04T22:41:40Z"
 content="""
Specifically I'm looking for a workaround that would make this behavior possible in this use case:

The user decides some file in the local repository are not needed anymore, they are not needed to be in any repository at all and wants to reclaim the space as soon as possible (or at least make the files available to 'unused' for later removal). Removing the content the from local repository (git-rm ?) the space can be immediately reclaimed in the local hard drive, without the need to connect and sync unavailable/far remotes. As the other remotes and old backups become available and sync, they delete the unused content as well (or at least make it available to 'unused').

Is there a way to achieve this?
"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="108.236.230.124"
 subject="comment 4"
 date="2014-06-05T17:35:31Z"
 content="""
`git annex drop --force $file; git rm $file`

Makes sense in some situations anyway..
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawmYMS6tQ8q9k1fylGyA9Q6fdFzRBotME2Q"
 nickname="tom"
 subject="comment 5"
 date="2014-06-06T09:57:32Z"
 content="""
ah, I get it. Thanks!
"""]]


code:
I used to save movies with the srt subtitle files next to them.

Usually vlc finds it because it's on the same directory than the movie file, however with git annex the link is located on another folder.
So after adding movies to git, the subtitles doesn't load anymore.

couldn't find a quick fix. I'm thinking a bash script, but wanted to discuss it here with all annex users.

I know It's out of annex scope, but I think a movie archive is a great scenario for git-annex.
most of my HD is filled up with movies from the camcorder, screencast, etc...
And we usually don't modify those files


code:
[[!comment format=mdwn
 username="http://joey.kitenet.net/"
 nickname="joey"
 subject="comment 1"
 date="2011-12-23T16:16:19Z"
 content="""
From what you say, it seems that vlc is following the symlink to the movie content, and then looking for subtitles next to the file the symlink points to. It would have to explicitly realpath the symlink to have this behavior, and this sounds like a misfeature.. perhaps you could point out to the vlc people the mistake in doing so?

There's a simple use-case where this behavior is obviously wrong, without involving git-annex. Suppose I have a movie, and one version of subtitles for it, in directory `foo`. I want to modify the subtitles, so I make a new directory `bar`, symlink the large movie file from `foo` to save space, and copy over and edit the subtitles from `foo`. Now I run vlc in `bar` to test my new subtitles. If it ignores the locally present subtitles and goes off looking for the ones in `bar`, I say this is broken behavior.
"""]]


code:
[[!comment format=mdwn
 username="http://joey.kitenet.net/"
 nickname="joey"
 subject="comment 2"
 date="2011-12-23T18:43:05Z"
 content="""
Since subtitle files are typically pretty small, a workaround is to simply check them into git directly, and only use git-annex for the movies. (Or `git annex unannex` the ones you've already annexed.)
"""]]


code:
I am struggling for so long time, everytime getting different errors :( :(

I digged up my backup and started from scratch. Still not working.

On the (linux) server (directory contains already files):

     cd bin
     git init
     git annex init server
     git annex direct
     git annex add .

Then, on the Windows client:

     $ git clone ssh://me@server/srv/data/bin bin
     Cloning into 'bin'...
     warning: remote HEAD refers to nonexistent ref, unable to checkout.

What's going on here?

EDIT: Of course, I would not ask if everything would work as expected. But the client repository in windows is always empty, no matter what I do (issueing git annex init, git add remote, git annex sync, ...) does not seem to help me ...

EDIT2: When I continue on the windows side (despite the warning):

     cd bin
     git annex init client
     git annex sync
     Permission denied (publickey,keyboard-interactive).
     (merging origin/git-annex origin/synced/git-annex into git-annex...)
     (Recording state in git...)

       Remote origin does not have git-annex installed; setting annex-ignore
     commit  ok
     pull origin Already up-to-date!
     Merge made by the 'recursive' strategy.

     ok
     push origin To ssh://me@server/srv/data/bin
        b79922f..4fe0505  git-annex -> synced/git-annex
        2f6a601..02a2603  annex/direct/master -> synced/master

     ok

But that's not true, git-annex is definitely installed on the server (it's just the Debian package and it resides in /usr/bin/git-annex). In any case, the client repository is still empty

EDIT3: Struggle, struggle, ...

Finally, for whatever reason, the client dir is not empty. What seems to be missing on the server was:

     git annex sync

but this is never written (also not in the walkthrough). Can anyone tell me if it's required and why?

Furthermore, even that I "see" at least the file references now, I am not able to get them:

     $ git annex get apps
     get apps/gitignore (not available)
       Try making some of these repositories available:
             10d0ffd8-f499-4b55-83e6-ff58cd98edc5 -- server

       (Note that these git remotes have annex-ignore set: origin)
     failed
     git-annex: get: 1 failed

That's an SSH remote, it *is* available! What is meant by that?


code:
[[!comment format=mdwn
 username="divB"
 ip="171.67.174.99"
 subject="comment 1"
 date="2014-05-10T21:20:36Z"
 content="""
I think, although not sure, that all of this is related of failing/inconsistent ssh calls.

I filed a bug here: https://git-annex.branchable.com/bugs/git-annex_ignores_GIT__95__SSH

The problem is that failure in ssh does not result in proper error messages in git-annex, resulting in inconsistent repositories.
"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="108.236.230.124"
 subject="comment 2"
 date="2014-05-16T16:49:09Z"
 content="""
After you add a file to a git repository, you have to git commit it. Using git-annex is no different. The walkthrough shows using `git commit`: <http://git-annex.branchable.com/walkthrough/adding_files/>; you can use `git-annex sync` if you prefer.

<pre>
Permission denied (publickey,keyboard-interactive).
 (merging origin/git-annex origin/synced/git-annex into git-annex...)
 (Recording state in git...)

   Remote origin does not have git-annex installed; setting annex-ignore
</pre>

git-annex uses a heuristic to determine if git-annex-shell is not installed on a remote server: It tries to run it, and if that fails, it tries to run git fetch. If that succeeds it assumes this means it can log into the server, but git-annex-shell is not installed.

Above, that heuristic seems to have failed; you seem to have given the wrong password or something so it didn't run git-annex-shell, but then it was able to log in for the git fetch.

You will need to run this command to fix it up: `git config remote.origin.annex-ignore false`
"""]]


code:
I did not recieve feedback on my comment [1], so I try to post my question again but more clearly.

Is it possible to run the assistent/the webapp without the functionality of 'git annex watch'?

I'd like to use the assistant and to have the automatic syncing but I do not want the local repository to be watched.  Instead I prefer to manually add/drop my files.

I do not see the 'pause button' mentioned in my earlier question [1].

[1] http://git-annex.branchable.com/forum/webapp_and_manual_mode/


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="4.154.0.63"
 subject="comment 1"
 date="2013-08-26T18:14:17Z"
 content="""
The assistant is currently only able to transfer files that it has added itself. So if you disable syncing, you have to manually upload any files you add.

I doubt that I will change this in the assistant, because supporting this use case would complicate it unnecessarily for a use case that is not what it's designed to do. It's more likely that `git annex sync` will get an option to also transfer file contents.
"""]]


code:
[[!comment format=mdwn
 username="ringprince"
 ip="134.76.140.110"
 subject="comment 2"
 date="2013-08-26T18:54:40Z"
 content="""
Thanks for the clarification.

Although

* not what I hoped for ;-)
* I don't see that it would get complicated
* I do not consider my use case special

Anyway, thanks for this great piece of software.
"""]]


code:
Somehow I seemingly have not understood the manual mode.

If I have a file in my repository, that is not yet in the annex (or in git), and I start the webapp, that file is automatically added to git-annex (and git).

Why is that?

And how can I use the webapp without such interference?


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="4.154.0.140"
 subject="comment 1"
 date="2013-07-21T17:17:58Z"
 content="""
Setting a repository to manual mode prevents the assistant from transferring new files into that repository. You have to manually run `git annex get`.

To prevent the assistant from automatically adding files, you can click the pause button next to the current repository in the webapp.
"""]]


code:
[[!comment format=mdwn
 username="ringprince"
 ip="134.76.2.1"
 subject="comment 2"
 date="2013-07-23T08:47:58Z"
 content="""
Not sure, what you mean.

Disabling sync?
But that also disables sync, right?  ;-)  So, if I add a file manually (via command line) that file is not synced automatically, correct?

I do not see any other 'pause' button next to the repository.

I am fine with git-annex doing the syncing automatically for me, but I want to be in control over the files git annex actually uses.

If I just downloaded a big file, I might want to check it first before having git-annex move it to other repositories.

"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawkAUMhKOSkh9JaBA6xst3XxQIIsDEq5Zd4"
 nickname="Ovidiu"
 subject="Also confused by the manual mode"
 date="2014-05-18T06:39:54Z"
 content="""
@Joeyh: would you mind explaining the modes maybe on their own wiki page?
I've got most of them but the sense and functioning of the manual method escapes me :-/
"""]]


code:
Sorry, I have again an issue :(

What could be the problem that the webapp is not starting (in Windows)? git annex just exists without messages, connection to localhost failes of course

    me@laptop /cygdrive/c/Data
    $ git annex --verbose --debug webapp

    me@laptop /cygdrive/c/Data
    $

If I run it within an annexed directory I get at least one message - but it still immideately exits:

    me@laptop /cygdrive/c/Data/annex
    $ git annex --verbose --debug webapp
    [2014-07-11 19:06:58 Pacific Daylight Time] chat: git-annex ["--verbose","--debug","webapp"]

    me@laptop /cygdrive/c/Data/annex
    $

    git-annex version: 5.20140707-g8116d10
    build flags: Assistant Webapp Webapp-secure Pairing Testsuite S3 WebDAV DNS Feeds Quvi TDFA CryptoHash
    key/value backends: SHA256E SHA1E SHA512E SHA224E SHA384E SKEIN256E SKEIN512E SHA256 SHA1 SHA512 SHA224 SHA384 SKEIN256 SKEIN512 WORM URL
    remote types: git gcrypt S3 bup directory rsync web webdav tahoe glacier ddar hook external
    local repository version: 5
    supported repository version: 5
    upgrade supported from repository versions: 2 3 4

EDIT:
1.) Interestingly, also "git annex test" fails:

    [...]
    ok
    push origin To C:/Data/.t\repo
       bcc4611..91f5218  git-annex -> synced/git-annex
       6f47922..92c578a  annex/direct/master -> synced/master

    ok
    add ../dir2/foo ok
    (Recording state in git...)
    FAIL
          Exception: .t/tmprepo9/.git/annex/journal: getDirectoryContents: permission denied (Access is denied.)

    2 out of 83 tests failed
      (This could be due to a bug in git-annex, or an incompatability
       with utilities, such as git, installed on this system.)

2.) There's really something very weird going on. At the beginning it worked. Then it stopped (as per above). Sometimes, when I start it using "Git Bash" it works. But then not any more.
3.) Rebooting the machine does not help

EDIT once again:
4.) I am afraid there is something big broken. Suddenly I could start the daemon again. The thing I did was to delete my complete annex.
But if I shutdown the daemon now I get:

    $ git annex webapp
    Launching web browser on file://C:\cygwin\tmp\webapp85796.html

      Detected a filesystem without fifo support.

      Disabling ssh connection caching.

      Detected a crippled filesystem.

      Enabling direct mode.

    (Recording state in git...)
    WebApp crashed: ExitFailu

... and the whole loop starts again. webapp daemon does not start as long as I delete the annex. Interestingly this is independent from %USERPROFILE%\.config. Deleting this directory does not change anything. So somehow git-annex "knows" that there is somewhere an annex and fails if it is.


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="209.250.56.2"
 subject="comment 1"
 date="2014-07-14T19:05:32Z"
 content="""
git-annex stores the locations of repositories in `.config/git-annex/autostart` in what passes for a home directory on Windows.

Are you sure it doesn't manage to at least open a web browser tab, possibly one with an error in it? This would be easy to miss if you have a lot of tabs open. I seem to have partially reproduced this, at least that's the behavior I see.

The root of the problem seems to be that it thinks the daemon is running, but it's not. So it just opens the web browser and then stops. After I deleted .git/annex/daemon*, and .git/annex/url and .git/annex/daemon.log, `git annex webapp` worked again.
"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="209.250.56.2"
 subject="comment 2"
 date="2014-07-14T20:06:03Z"
 content="""
I think that I have fixed this problem. The Windows autobuilder should produce a fixed build within an hour.
"""]]


code:
[[!comment format=mdwn
 username="divB"
 ip="128.12.90.218"
 subject="comment 3"
 date="2014-07-15T05:43:35Z"
 content="""
Thank you, really really great. This part seems to work again.
For some reason, the assistant takes the wrong (?) git.exe (as I described here: http://git-annex.branchable.com/forum/Restricting_SSH_+_supply_key/)
"""]]


code:
To start the webapp on a X-less server it's possible to use the parameter `--listen` and then connect from a client with X to it.

How can I start the webapp on all annex repos on this server like on a client with `assistant --autostart`? Would be nice to start the assistant on all configured repos which listens on a defined port, f.e. `git annex assistant --autostart --listen 0.0.0.0:8888`. It would be even nicer to have a parameter to chose a autostart configuration file, f.e. `git annex assistant --autostart --autostart-config /etc/git-annex/autostart --listen 0.0.0.0:8888`, so this could end up in a real server application, a init script would be easy to write...


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 nickname="joey"
 subject="comment 1"
 date="2013-05-13T18:49:25Z"
 content="""
Well, it would be possible to add a configuration setting in the repository that tells the assistant what address:port to listen on.

But maybe it would be better to not use `assistant --autostart` in your situation? That command is basically the same as this shell script:

[[!format sh "
#!/bin/sh
for dir in $(cat $HOME/.config/git-annex/autostart); do
    cd $dir
    git annex webapp &
done
"]]

So you can write similar shell scripts that start the webapp with whatever options you like.
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawmiqeXJtP04fzHOjXs17kHO33v7dWR2xwA"
 nickname="Jaco"
 subject="Init Script"
 date="2013-11-23T08:28:32Z"
 content="""
Hi Joey,

Could you help out with writing an init.d script to safely start and stop the webapp on a headless server?
I made an attempt below based on examples from the internet, but have no idea if it will work.

	#!/bin/bash
	# git-annex
	# chkconfig: 345 20 80
	# description: Git Annex WebApp startup and shutdown script.
	# processname: git-annex

	DAEMON=git-annex webapp

	NAME=git-annex
	DESC=\"Git Annex WebApp init script\"
	PIDFILE=/var/run/$NAME.pid
	SCRIPTNAME=/etc/init.d/$NAME

	case \"$1\" in
	start)
		printf \"%-50s\" \"Starting $NAME...\"
		for dir in $(cat $HOME/.config/git-annex/autostart); do
			cd $dir
			PID=`$DAEMON > /dev/null 2>&1 & echo $!`
			#echo \"Saving PID\" $PID \" to \" $PIDFILE
		        if [ -z $PID ]; then
	        		printf \"%s\n\" \"Fail\"
		        else
	        		echo $PID > $PIDFILE
				printf \"%s\n\" \"Ok\"
			fi
		done
	;;
	status)
	        printf \"%-50s\" \"Checking $NAME...\"
	        if [ -f $PIDFILE ]; then
			for PID in $(cat $PIDFILE); do
				if [ -z \"`ps axf | grep ${PID} | grep -v grep`\" ]; then
					printf \"%s\n\" \"Process dead but pidfile exists\"
				else
					echo \"Running\"
				fi
			done
	        else
	            printf \"%s\n\" \"Service not running\"
	        fi
	;;
	stop)
	        printf \"%-50s\" \"Stopping $NAME\"
		if [ -f $PIDFILE ]; then
			for PID in $(cat $PIDFILE); do
				kill -HUP $PID
				printf \"%s\n\" \"Ok\"
			done
			rm -f $PIDFILE
	        else
	            printf \"%s\n\" \"pidfile not found\"
	        fi
	;;
	restart)
	  	$0 stop
	  	$0 start
	;;
	*)
	        echo \"Usage: $0 {status|start|stop|restart}\"
	        exit 1
	esac
"""]]


code:
I have two repositories: A(direct) and B(indirect). At some point, I performed the following steps:

1. Add a file in A.
2. Sync A.
3. Sync B, but not get all files.
4. In B, remove a file F that was newly added in A, using `git rm` and `git commit`.
5. Now get all files from A.

This results in all "current" files being copied from A to B, but not file F.

If I checkout a branch in B for an older commit which still had file F, I can see the symlink created in the branch. But now if I try to get the file in B from A, annex says that the remote is not available:

    sameerds@gajanan:/mnt/mandos/test$ git annex get
    get IMG_0570.JPG
      Unable to access these remotes: A

      Try making some of these repositories available:
      	530e4f81-474d-4b0b-9f13-4beb7361302c -- A
    failed
    git-annex: get: 1 failed

The remote for A exists in the listing for `git remote`, and `git annex sync` does not make this error message go away. If I checkout `master` again, then everything works fine.

But of course, this a different problem from my original one. It would a way to bring all files from the remote, that are ever used by any commit reachable in the local git history, irrespective of the currently checkout out commit. Perhaps an option `--deep-copy` to the standard `git annex get`.


code:
[[!comment format=mdwn
 username="http://sameer.sbuddhe.net/blog/"
 nickname="sameerds"
 subject="never mind ..."
 date="2013-12-31T04:06:43Z"
 content="""
... I discovered the [[`--all` option|http://git-annex.branchable.com/forum/Moving_older_version__39__s_file_content_without_doing_checkout/]] which does just what I was looking for!

"""]]


code:
Hi

I have a laptop, server and a regular PC. Laptop and PC are using same "backup" repo on server. Thay all get notified via ssh. My goal is to have synchronization and backup in one remote repo. When I wan to list all files on all repos i do:

    git-annex whereis -A (some files do fail. Don't know why)

But that displays hashes. Not file names. How can to list file names on all repos instead of hashes ?


code:
[[!comment format=mdwn
 username="http://ypid.wordpress.com/"
 ip="213.153.84.215"
 subject="-A  --all      operate on all versions of all files"
 date="2014-07-07T07:42:03Z"
 content="""
Hi

`-A` operates on all the files even on files which are not accessible from your current working directory. If you omit the `-A` then filenames will be printed.
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawkNE-H4vEcbcGndxq5daT8qUb7yIf7r1OE"
 nickname="Łukasz"
 subject="comment 2"
 date="2014-07-07T08:40:57Z"
 content="""
Yes they will be omitted. I just want to list files that are on my laptop (for example) and on remote repo.
"""]]


code:
[[!comment format=mdwn
 username="http://ypid.wordpress.com/"
 ip="213.153.84.215"
 subject="git annex whereis --in origin"
 date="2014-07-07T09:45:48Z"
 content="""
So you only want to see whereis for files which are present on certain annexes? You can use `git annex whereis --in e64553d0-779a-4c79-939a-d5b7f3e56a57` or `git annex whereis --in origin` (modify the repo names).

Have a look on [QUERY COMMANDS](/git-annex/).
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawkNE-H4vEcbcGndxq5daT8qUb7yIf7r1OE"
 nickname="Łukasz"
 subject="comment 4"
 date="2014-07-07T10:05:06Z"
 content="""
git-annex whereis --not --in here

Does not display anything.

git-annex whereis --not --in here -A

Displays files on remote only. Every file has an additional message (0 copies) failed. This displays hash names. How to display file names ? I am 100% sure that my remote repo has some files that are not present on laptop or PC.
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawkNE-H4vEcbcGndxq5daT8qUb7yIf7r1OE"
 nickname="Łukasz"
 subject="comment 5"
 date="2014-07-07T10:06:03Z"
 content="""
One more thing. I am using indirect mode with assistant. If that matters in some way.
"""]]


code:
Any progress on Windows port? That would be very nice to have!
Depending on the scale of it, I might be able to help.


code:
[[!comment format=mdwn
 username="http://joey.kitenet.net/"
 nickname="joey"
 subject="comment 1"
 date="2012-03-12T06:43:02Z"
 content="""
[[todo/windows_support]] has everything I know about making a windows port. This badly needs someone who understand Windows to dive into it. The question of how to create a symbolic link (or the relevant Windows equivilant) from haskell on Windows
is a good starting point..
"""]]


code: Know what'd be a sweet feature?  A file explorer in the webapp that lets you get and drop files (for a Manual local repository).  Especially when the webapp becomes available on Android.  I'd love to be able to select what is and isn't present on a small device by some means other than moving files around.


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 nickname="joey"
 subject="comment 1"
 date="2013-04-11T14:55:43Z"
 content="""
There's a page called [[design/assistant/partial_content]] that discusses this idea and related things.
"""]]


code:
I'm trying to distribute a large annex to a number of smaller archive drives.

While copying to a directory special remote, the current behaviour is to continue trying copying files into a remote, even as diskspace there has been exhausted.
It would make sense for git-annex copy to actually stop instead.


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="4.154.0.140"
 subject="comment 1"
 date="2013-07-18T19:49:43Z"
 content="""
Hmm, it might make sense to stop if there is no more space at all. However, just because the first file doesn't fit doesn't mean several following, smaller files can't be copied into the remaining space.

Since it checks the free space before actually copying anything, it's not like there's much overhead in running through the list of files it was asked to copy and trying to copy them all, either.

So, I'm unconvinced on this one.
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawln3ckqKx0x_xDZMYwa9Q1bn4I06oWjkog"
 nickname="Michael"
 subject="comment 2"
 date="2013-07-18T20:22:05Z"
 content="""
Actually with 4.20130709, I see the drive (ntfs-3g under Linux, gpg-encrypted special remote) being filled to 0 remaining space, even with reservespace of 300mb.  Guess it's another bug anyway.
"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="4.154.0.140"
 subject="comment 3"
 date="2013-07-20T20:33:37Z"
 content="""
Indeed, that was a bug with encrypted directory special remotes. Fixed it.
"""]]


code:
Is it possible to use git-annex without having [[location tracking]] commits in the style of "got a video I want to rewatch on the plane" or "freed up space" in the main tree?

I consider these changes to be volatile, and irrelevant to the archive history. While they are unproblematic when it comes to merging, they make the commit tree rather complicated, especially with multiple users (as opposed to a single user managing his files on an external disk, a server and his laptop). Some users might even want to contribute to a shared repository without reporting on what they checked out.

As a minimal solution, I configured a repository to ``.gitignore`` ``.git-annex/*:*.log``, but even when using modes that do not require that information (``git annex copy --from`` instead of ``git annex get``), that failes when git-annex tried to git-add ignored files.

A more elaborate solution might be to keep location tracking information in a branch on its own (as suggested in [[todo/branching]]), keeping the main tree clean of such commits. A stealth user could then configure that branch to never be pushed. (Alternatively, if git-annex respects .gitignore and doesn't try to check in changes on ignored files, he could locally ``.gitignore`` ``.git-annex/*:*.log``.)

> A stealth user can simply avoid pushing, and so keep their repository
> in a forked state, that can still pull changes from origin.
>
> Beyond that, [[todo/branching]] is the best solution.
>
> I don't think that gitignoring the log files is a good plan, because
> if the files are left modified and uncommitted, git will not be able to
> merge other changes it pulls. The automerging of log files only works
> if any local changes to them have been committed.
>
> It would be possible to add a knob that
> simply blocks all local modifications to the log files. --[[Joey]]


code:
I am running git-annex on Mac OS 10.9 and syncing to a Linux device that was attached to a Synology NAS.  Both were running December versions of git-annex.

While working on an application in IntelliJ my workspace.xml file disappeared.  It went from being a real file to being a symlink to a location that didn't exist.

The symlink looks like this: workspace.xml -> ../../../.git/annex/objects/M2/8W/SHA256E-s68106--90e9cc4f617c9034db1bf462d058b82c59ade0be58de1d3a3e2f8c02606631c2.xml/SHA256E-s68106--90e9cc4f617c9034db1bf462d058b82c59ade0be58de1d3a3e2f8c02606631c2.xml

I tried git-annex fsck and I get this message: ** No known copies exist of XXXXXXX/workspace.xml

"XXXXXXX" is a path I needed to obscure I can see it in the git-annex repo but it is a reference to a symlink that doesn't exist

It was mentioned in another thread ( http://git-annex.branchable.com/forum/git_annex_on_osx_only_creating_symlinks__63____63__/ ) that this could happen if I did a forced drop but I do not use git-annex on the command line.  I'm using the assistant and having it manage all of my files for me.

I tried looking at the log for it and it shows this:

<pre>
timmattison$ /Applications/git-annex.app/Contents/MacOS/git-annex log XXXXXXX/.idea/workspace.xml
- 2014-01-03 19:49:58 XXXXXXX/.idea/workspace.xml | d6747880-b355-4d41-b4e5-d1ad3afcb4a5 -- timmattison@MacBook-Pro.local:~/Desktop/annex
+ 2014-01-03 17:08:32 XXXXXXX/.idea/workspace.xml | d6747880-b355-4d41-b4e5-d1ad3afcb4a5 -- timmattison@MacBook-Pro.local:~/Desktop/annex
(Recording state in git...)
</pre>

So I can see that the git-annex thinks the file was deleted but what really happened is that IntelliJ just modified it.  workspace.xml files are modified very often when working in IntelliJ.

Is this a known bug that has been fixed in the new version (2013-12-30 or later)?

Thanks,
Tim


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="209.250.56.35"
 subject="comment 1"
 date="2014-01-06T18:38:51Z"
 content="""
If the file was modified, git-annex should notice and add the new version. So the mere act of modifying the file does not explain how you ended up with a symlink to the old version of the file. Although it does explain why git-annex log would show that the old version of the file had been deleted.

What does `git log` say when run on the workspace.xml file?

"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawkipQLNyt8RHREHpg2k5wdYeRSCCvSNSBg"
 nickname="Tim"
 subject="git log for workspace.xml"
 date="2014-01-06T19:17:13Z"
 content="""
git log on the file gives a ton of output, is this what you need?

commit 93157022d8ee57097db29fcc669e99e218b54657
Author: Tim Mattison <tim.mattison@email.com>
Date:   Fri Jan 3 17:08:03 2014 -0500

commit 04dab5520f33140a21882515b4036f71729a5f89
Author: Tim Mattison <tim.mattison@email.com>
Date:   Fri Jan 3 17:06:54 2014 -0500

commit b75be90bf91c9f9314e567d5a28e4497070de9af
Author: Tim Mattison <tim.mattison@email.com>
Date:   Fri Jan 3 16:58:24 2014 -0500

commit 4045b2d7a71b9ab692c20eeab73bf196a280940d
Author: Tim Mattison <tim.mattison@email.com>
Date:   Fri Jan 3 16:58:05 2014 -0500

commit 25de8a1fcd5cce53b8077443dccba1591255431f
Author: Tim Mattison <tim.mattison@email.com>
Date:   Fri Jan 3 16:57:32 2014 -0500

commit dd118ae88cdccb5a7d68a7ae14033304865cfc4e
Author: Tim Mattison <tim.mattison@email.com>
Date:   Fri Jan 3 16:53:37 2014 -0500

commit eaa5f85b7feed8ba17e3874763b4d7aa6e05b5c1
Author: Tim Mattison <tim.mattison@email.com>
Date:   Fri Jan 3 16:53:23 2014 -0500

commit 4d310d7d16234a1a6bcf0f14ebae232c57d904a7
Author: Tim Mattison <tim.mattison@email.com>
Date:   Fri Jan 3 16:52:42 2014 -0500

commit 2487ae7f99c7b81c1648fdc88d0b39a1cf6e4d03
Author: Tim Mattison <tim.mattison@email.com>
Date:   Fri Jan 3 16:52:21 2014 -0500

commit 9e0040311c43e6e1f04ff3b8fdd3ac5570c8f2d1
Author: Tim Mattison <tim.mattison@email.com>
Date:   Fri Jan 3 16:51:54 2014 -0500
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawkipQLNyt8RHREHpg2k5wdYeRSCCvSNSBg"
 nickname="Tim"
 subject="Formatting!"
 date="2014-01-06T19:22:36Z"
 content="""
I swear, some day I'll get the formatting correct.  Sorry!
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawkipQLNyt8RHREHpg2k5wdYeRSCCvSNSBg"
 nickname="Tim"
 subject="Just happened again"
 date="2014-01-16T16:23:00Z"
 content="""
This happened again and now I have this:

lrwxr-xr-x  1 timmattison  staff  197 Jan 16 11:18 distribute-client-applications.sh -> ../../../.git/annex/objects/JV/91/SHA256E-s921--51fc6a6e2924ff6e2673971b5786fe0f204a5984a700a2241cc77383f04f544e.sh/SHA256E-s921--51fc6a6e2924ff6e2673971b5786fe0f204a5984a700a2241cc77383f04f544e.sh

This was on the machine that had the file, not a remote machine it was syncing to.  I was editing the script, I ran it, the bash script segfaulted (expected), and then git-annex essentially deleted the file.  Can you help me understand why this happens?  It's obvious when it is a file I'm working on but I'm afraid it could be happening to other files and I don't know about it.
"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="209.250.56.146"
 subject="comment 5"
 date="2014-03-06T18:22:31Z"
 content="""
Do you edit this file in multiple places? This could be an occurance of this bug: [[bugs/direct_mode_merge_can_overwrite_local,_non-annexed_files]] which is fixed in the latest release.

If the program that writes the workspace.xml file did so by first deleting it, and then writing the new version, this could result in the assistant committing the deletion, which makes the new version a local, non-annexed file, and then if a pull is received that modified the file, I think the above bug could happen.

You could tell if this was the case by looking at the git log of the directory containing the file, and see if it has been repeatedly deleted and added back to the repository. The git log snippet you pasted unfortunately does not let me tell this information.
"""]]


code:
Hello people :)

I'm learning to use git annex and I'm doing some tests to know how it works from an user point of view... My big goal would be to use git annex to track my entire /home directory in direct mode, so I would be able to sync my main computer with my laptop in a really effective way. But of course I feel some fear because things could go wrong and of course I have lots of important files I can't lose —nothing really important since I'll backup that /home in an external hard drive, but it would make me cry if I have to spend some hours restoring my /home :S

So, do you think that tracking all the /home would be a reasonable goal?

Thanks a lot for your time :)


code:
[[!comment format=mdwn
 username="https://me.yahoo.com/a/FHnTlSBo1eCGJRwueeKeB6.RCaPbGMPr5jxx8A--#ce0d8"
 nickname="Hamza"
 subject="comment 1"
 date="2013-12-09T06:07:00Z"
 content="""
I do track my home dir using annex, but instead of one giant annex I have multiple annex repos. It is much faster than one giant annex and when used in combination with mr you can threat all repos as one.
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawleVyKk2kQsB_HgEdS7w1s0BmgRGy1aay0"
 nickname="Milan"
 subject="comment 2"
 date="2013-12-09T11:16:48Z"
 content="""
what is 'mr'?
"""]]


code:
[[!comment format=mdwn
 username="https://me.yahoo.com/a/FHnTlSBo1eCGJRwueeKeB6.RCaPbGMPr5jxx8A--#ce0d8"
 nickname="Hamza"
 subject="comment 3"
 date="2013-12-09T14:56:13Z"
 content="""
It [1] allows you to run commands on a set of repositories.

[1] http://myrepos.branchable.com/
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawlqimC5gbxsILaBlnuVXYtfSMDIhiyvHfw"
 nickname="Valentín"
 subject="comment 4"
 date="2013-12-10T19:34:06Z"
 content="""
Hi Hamza :)

I like your way of achieving the same goal —and existing ``mr``, which I hadn't known until now, it looks like the best alternative.

Thanks a lot for your time :)
"""]]


code:
Imagine putting a git-annex drive in a time capsule. In 20, or 50, or 100
years, you'd like its contents to be as accessible as possible to whoever
digs it up.

This is a hard problem. git-annex cannot completely solve it, but it does
its best to not contribute to the problem. Here are some aspects of the
problem:

* How are files accessed? Git-annex carefully adds minimal complexity
  to access files in a repository. Nothing needs to be done to extract
  files from the repository; they are there on disk in the usual way,
  with just some symlinks pointing at the annexed file contents.
  Neither git-annex nor git is needed to get at the file contents.

  (Also, git-annex provides an "uninit" command that moves everything out
  of the annex, if you should ever want to stop using it.)

* What file formats are used? Will they still be readable? To deal with
  this, it's best to stick to plain text files, and the most common
  image, sound, etc formats. Consider storing the same content in multiple
  formats.

* What filesystem is used on the drive? Will that filesystem still be
  available? Whatever you choose to use, git-annex can put files on it.
  Even if you choose (ugh) FAT.

* What is the hardware interface of the drive? Will hardware still exist
  to talk to it?

* What if some of the data is damaged? git-annex facilitates storing a
  configurable number of [[copies]] of the file contents. The metadata
  about your files is stored in git, and so every clone of the repository
  means another copy of that is stored. Also, git-annex uses filenames
  for the data that encode everything needed to match it back to the
  metadata. So if a filesystem is badly corrupted and all your annexed
  files end up in `lost+found`, they can easily be lifted back out into
  another clone of the repository. Even if the filenames are lost,
  it's possible to [[tips/recover_data_from_lost+found]].


code:
# NAME

git-annex-shell - Restricted login shell for git-annex only SSH access

# SYNOPSIS

git-annex-shell [-c] command [params ...]

# DESCRIPTION

git-annex-shell is a restricted shell, similar to git-shell, which
can be used as a login shell for SSH accounts.

Since its syntax is identical to git-shell's, it can be used as a drop-in
replacement anywhere git-shell is used. For example it can be used as a
user's restricted login shell.

# COMMANDS

Any command not listed below is passed through to git-shell.

Note that the directory parameter should be an absolute path, otherwise
it is assumed to be relative to the user's home directory. Also the
first "/~/" or "/~user/" is expanded to the specified home directory.

* configlist directory

  This outputs a subset of the git configuration, in the same form as
  `git config --list`. This is used to get the annex.uuid of the remote
  repository.

  When run in a repository that does not yet have an annex.uuid, one
  will be created, as long as a git-annex branch has already been pushed to
  the repository.

* inannex directory [key ...]

  This checks if all specified keys are present in the annex,
  and exits zero if so.

* dropkey directory [key ...]

  This drops the annexed data for the specified keys.

* recvkey directory key

  This runs rsync in server mode to receive the content of a key,
  and stores the content in the annex.

* sendkey directory key

  This runs rsync in server mode to transfer out the content of a key.

* transferinfo directory key

  This is typically run at the same time as sendkey is sending a key
  to the remote. Using it is optional, but is used to update
  progress information for the transfer of the key.

  It reads lines from standard input, each giving the number of bytes
  that have been received so far.

* commit directory

  This commits any staged changes to the git-annex branch.
  It also runs the annex-content hook.

* notifychanges

  This is used by `git-annex remotedaemon` to be notified when
  refs in the remote repository are changed.

* gcryptsetup gcryptid

  Sets up a repository as a gcrypt repository.

# OPTIONS

Most options are the same as in git-annex. The ones specific
to git-annex-shell are:

* --uuid=UUID

  git-annex uses this to specify the UUID of the repository it was expecting
  git-annex-shell to access, as a sanity check.

* -- fields=val fields=val.. --

  Additional fields may be specified this way, to retain compatability with
  past versions of git-annex-shell (that ignore these, but would choke
  on new dashed options).

  Currently used fields include remoteuuid=, associatedfile=,
  and direct=

# HOOK

After content is received or dropped from the repository by git-annex-shell,
it runs a hook, `.git/hooks/annex-content` (or `hooks/annex-content` on a bare
repository). The hook is not currently passed any information about what
changed.

# ENVIRONMENT

* GIT_ANNEX_SHELL_READONLY

  If set, disallows any command that could modify the repository.

  Note that this does not prevent passing commands on to git-shell.
  For that, you also need ...

* GIT_ANNEX_SHELL_LIMITED

  If set, disallows running git-shell to handle unknown commands.

* GIT_ANNEX_SHELL_DIRECTORY

  If set, git-annex-shell will refuse to run commands that do not operate
  on the specified directory.

# SEE ALSO

[[git-annex]](1)

git-shell(1)

# AUTHOR

Joey Hess <joey@kitenet.net>

<http://git-annex.branchable.com/>

Warning: Automatically converted into a man page by mdwn2man. Edit with care