code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawnsuhFUIfWNT-Q-C02FDaSQqceFDge5M9w"
 nickname="Florian"
 subject="comment 2"
 date="2012-10-10T10:48:04Z"
 content="""
Installing it properly seems to be quite a bit of effort, given the long list of Haskell dependencies. Most of the machines I plan to use it on are not under my control and seem to be somewhat outdated. What way of installing it properly would you suggest?
"""]]


code:
[[!comment format=mdwn
 username="http://colberg.org/"
 ip="2001:470:1d:4d4:2d4b:7654:2c00:b95c"
 subject="comment 3"
 date="2012-11-12T05:05:39Z"
 content="""
Hi Florian,

I maintain a makefile for GNU/Linux systems that builds git-annex from scratch using a single command.

    git clone http://git.colberg.org/packages.git

    nice make -f packages.mk CONCURRENCY_LEVEL=16 PREFIX=$HOME/usr/rhel6-x64_64 install-git-annex

The makefile fetches pre-compiled GHC 6.x (x86_64), which works on old systems (e.g. RHEL 5), builds a recent GHC 7.x from source, a minimal Haskell environment with cabal-install, and finally git-annex. I have succesfully compiled git-annex on RHEL 5.x and 6.x, and CentOS 6.x.

If the build fails using the system's GCC, there is also a newer GCC in the same makefile:

    nice make -f packages.mk CONCURRENCY_LEVEL=16 PREFIX=$HOME/usr/rhel6-x64_64 install-gcc

Regards,
Peter
"""]]

repo_name: kakafarm/git

license: unknown


code:
Is there a way to sync all (or a subset of) local branches, not just the currently checked out branch?

Does this make sense at all, or does this show I am missing some important point in git-annex?

I am asking because I would like to use git-annex to keep git repositories with normal git files (with versioned and branched content) in sync.

If it's not currently possible, could you provide some pointers on where to start, if I wanted to change to Haskell source?

Thnx


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawlgyVag95OnpvSzQofjyX0WjW__MOMKsl0"
 nickname="Sehr"
 subject="comment 1"
 date="2012-10-29T17:04:30Z"
 content="""
Ok, this is what I have come up with:

Merges can only be done on checked out branches. That's why the only way to do this without breaking the whole idea of git-annex (sync) is to check out each branch and run it separately.

What would be the easiest way to do this? Considering possible unsaved changes in worktree and index?

Maybe simply using git-clone /tmp locally, which uses hardlinks for .git/objects/ which should make it perform ok?

"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawlgyVag95OnpvSzQofjyX0WjW__MOMKsl0"
 nickname="Sehr"
 subject="comment 2"
 date="2012-10-29T18:52:59Z"
 content="""
It turns out, this is even easier:

As the behavior of \"git-annex sync\" includes performing a \"git commit -a\", any pending changes in the work tree will be committed anyway. This means, that it is safe to simply checkout and sync any branch, one wants to sync.
"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="4.152.108.194"
 subject="comment 3"
 date="2012-11-04T20:08:02Z"
 content="""
Yes, that's right, `git annex sync` handles the current branch, and you can use it on different branches if desired.

(It is actually possible to merge non-checked out branches, but I have not implemented that for sync.)
"""]]


code:
I don't know if this is an "unusual setup" or something that should be supported, but it seems like a reasonable use case to me, so here goes:

I have three systems: a desktop, a laptop, and a little netbook.  I have one main annex that is currently synced between the desktop and the laptop at ~/annex on both, and to a secondary, Full Backup repo on the desktop, under /mnt.

I have some music on my laptop, and I like to sync ~/Music between the laptop and the netbook.  So I thought I'd make a new git-annex repo at ~/Music on both and sync them.  I want this to be kept separate from ~/annex.

I also thought it would be good to backup the music, so I thought I'd connect laptop:~/Music to the Full Backup repo at desktop:/mnt.  So I paired the repo at laptop:~/Music to the repo at desktop:/mnt.

So I had set up these repos:

* desktop:
  * /mnt  (Full Backup)
  * ~/annex  (Client)
* laptop:
  * ~/annex  (Client)
  * ~/Music  (Client)
* netbook:
  * ~/Music  (Client)

As I feared--but I wasn't sure, since I'm new to git-annex--since the desktop:/mnt repo was connected to laptop:~/Music, and desktop:~/mnt was also connected to desktop:~/annex,  this caused everything in laptop:~/Music to be synced to desktop:~/annex, because they were part of the same "set" of repos; they were all in the same .git/config files.

So now I have a few problems:

1.  How do I get all these music files out of my ~/annex repos?  I tried removing the remote repos from ~/annex/.git/config and then running 'git-annex unused', but that didn't result in it thinking all the music files were unused.  Do I have to delete them all manually?  (There are a *lot* of directories from ~/Music now in ~/annex).

2.  It seems that desktop:~/annex/.git is...now over 600 MB, even though few of the actual music files have synced over.  Is it possible to clean out ~/annex/.git and restore it to the pre-mess state without starting from scratch?

3.  Is it possible--or reasonable--to use a single Full Backup repo for multiple independent client repos?  That is, is it possible to do that without the independent client repos "cross-pollinating"?  Or do I need to use separate Full Backup repos for each "set" of client repos?

4.  I'm not sure if this is a separate problem, but I just want to make sure I understand the best way to handle things like this.  I wanted to disconnect desktop:~/annex from laptop:~/Music, but I thought that if I used the webapp to "Delete" the laptop:~/Music repo, it would actually start deleting the music files in laptop:~/Music, after moving them to desktop:~/annex.  Am I correct that that would have happened?  If so, "Deleting" a repo seems like a dangerous operation that should warn and prompt the user.  Maybe an "unlink repo" option needs to be available to simply disconnect two repos and leave their respective contents intact.

I apologize if these are newbie problems.  Despite reading every dev blog entry--both before and after the switchover--I'm still new to actually using git-annex.  I'm not sure if it's supposed to work this way or if I'm using it wrong.

If I am using it wrong...well, I can imagine other git-annex newbies doing similar things and ending up with similar messes, so maybe this needs to...be considered or something.  :)  Since I'm still experimenting with git-annex, I haven't put any "live" data in it, so I can blow away all the annexes and start over, but I would like to learn how to properly fix a situation like this in case I goof up in the future.

Thanks for your help.


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="209.250.56.47"
 subject="comment 1"
 date="2013-11-05T16:14:34Z"
 content="""
1. Turn off the assistant, switch to indirect mode (`git annex indirect`), use `git log --stat` to find the commit(s) that added the music files, and `git revert` the commits, in reverse chronological order. Or delete the files manually.
2. You have merged two git repositories. Without some serious git repository surgery (ie, resetting to an old version and deleting the reflog), the full history of both is going to remain stored in git going forward. However, a simple `git gc` might pack that down to a much smaller amount of disk space.
3. Combining two repositories merges their git histories. You then have two clones of one repository containing the full contents and git history of both. Don't do that if that's not what you want. The webapp is always careful to tell you when an action will combine two repositories.
4. Once you have combined two repositories, you have two synced repositories that contain the same data. Deleting one of them will just delete one copy of the data. It won't result in data loss, but neither will it untangle the two git trees that were previously combined.
"""]]


code:
Hi,

I am looking into using git-annex to sync my Music between 3 sources: desktop, laptop, backup, but I'm not sure if this will work ok.

I notice on the wiki that:  "Normally, the content of files in the annex is prevented from being modified. That's a good thing, because it might be the only copy, you wouldn't want to lose it in a fumblefingered mistake."

But most music players will download artwork, and some of course will dynamically modify/rename files according to meta data.  And I may be downloading and/or playing music on both the desktop and laptop.  So obviously I wouldn't want to be manually unlocking individual files here.  It would kinda interrupt the music listening experience.  So how would this work?

I am assuming that I would always have to do a git annex add *.mp3 *.flac *.jpg before doing a git annex sync.  But can I just keep everything unlocked?  Any side effects here?  Will this work or should I just use rsync?

Regards,

bk


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 nickname="joey"
 subject="comment 1"
 date="2013-04-18T20:01:50Z"
 content="""
You can use [[direct_mode]] to keep all files unlocked all the time.

You can use the [[assistant]] to automatically commit whatever changes are made to the directory, as well as automatically sync them to your other computers.
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawmgXUaPZHcAY8Wv4sTnX88CbJfYdYdKO-Y"
 nickname="Boyd"
 subject="comment 2"
 date="2013-04-18T23:33:10Z"
 content="""
Thanks Joey!  Anyway I see that with Fedora, I don't have the 'assistant'.  I don't even seem to have the 'direct mode'.  Also tried to do a cabal install git-annex, but ran into dependency hell.  Oh well, this is goin on the back burner for a while.  Looks very promising though!

Boyd

[root@zbox Music]# git annex direct
git-annex: Unknown command 'direct'

Did you mean one of these?
	reinject

[root@zbox Music]# git annex version
git-annex version: 3.20120615
local repository version: 3
default repository version: 3
supported repository versions: 3
upgrade supported from repository versions: 0 1 2
[root@zbox Music]#

"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 nickname="joey"
 subject="comment 3"
 date="2013-04-19T18:05:49Z"
 content="""
Fedora has an old version of git-annex. You can use the [[install/Linux_standalone]] tarball.
"""]]


code: I am looking to use git-annex as a library/tool to bring syncing to another application. Is there any documentation or open examples of doing something like this?


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawlaQ9Hd5o_wjCIcfC9MAx8GRli5-LyBang"
 nickname="George"
 subject="comment 1"
 date="2013-06-29T16:07:45Z"
 content="""
I should say specifically interfacing with git-annex
"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="4.154.4.193"
 subject="comment 2"
 date="2013-06-30T18:08:52Z"
 content="""
I think there are two approaches to doing this.

1. Use the git-annex command line interface from your program. You can use the --json switch to enable machine-parsable output of many git-annex commands. If something needs work to be more suitable to be used as \"plumbing\" in this way, we can improve it to meet your needs.

2. Use the git-annex Haskell code as a library for your program. The git-annex assistant is a great example of how far you can take this. It has the benefit that by accessing git-annex's internals, you can sometimes do things more efficiently than by using the CLI. Much of git-annex's code is already well modularized and suitable for use as a library in this way. The build system doesn't currently spit out git-annex libraries, but it would not be hard to make it do so. Of course this would entail writing at least some of your program in Haskell.
"""]]


code:
[[!comment format=mdwn
 username="Alexcei"
 ip="92.255.239.77"
 subject="comment 3"
 date="2014-03-25T05:43:36Z"
 content="""
With version 5.20140127 first approach became impossible. Why did you decide to remove json supported by the majority of command?
"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="209.250.56.41"
 subject="comment 4"
 date="2014-03-26T18:31:30Z"
 content="""
I didn't. I removed the --json options from commands that didn't necessarily output valid json when used with it. There are numerous commands that support --json (find, whereis, info, status, metadata, examinekey), and I'm open to adding it to other commands -- but it's probably not feasible to make commands like `get` output json, since their output includes progress info printed by rsync etc.
"""]]


code:
After installing the binary for Mac OS X and including some larger directories, my computer now seems quite busy (for the last days) with the import process. I see a process called git-annex causing the load, so it seems OK. As the git-annex assistant seems to hang quite a bit, I would like to see the progress using a command line interface. However, I cannot use the "git annex" command as I get the message "git: 'annex' is not a git command. See 'git --help'."

I guess this is my normal git version, installed by homebrew (the OS X version for apt-get) that does not know anything about the installed binary. Can I still use the CLI with this version? Or better, is there or will there be a way to install git-annex from source, or even better using "brew install git-annex"?


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawnaZYaspvzx1lQiM56UQo-X82BnPiAaiEY"
 nickname="Martin"
 subject="comment 1"
 date="2013-08-19T18:07:04Z"
 content="""
One more question. When running the git-annex binary the webapp opens. After closing the window I cannot access the webapp anymore. Using the IP + port number (http://127.0.0.1:53027/) does not work. Would be handy to have a bookmark for the webapp that works as long as the assistant is running.
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawnaZYaspvzx1lQiM56UQo-X82BnPiAaiEY"
 nickname="Martin"
 subject="git-annex via comand line"
 date="2013-08-19T21:53:11Z"
 content="""
Just found the page how to install the CLI on OS X: http://git-annex.branchable.com/install/OSX/
That solved all my questions for now.
"""]]


code:
Hallo,

I have got a Samsung Galaxy Note 10.1 I want to sync with a remote ssh-Server, but the tablet only supports exFAT and FAT file Systems, so I cannot use symlinks.

Does any solution exist for this?

Mebus


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="64.134.31.139"
 subject="comment 1"
 date="2013-10-22T17:37:44Z"
 content="""
The sad state of affairs that in the supposedly modern 21st century, popular operating systems are regressing in functionality?

Yes, git-annex can deal with this. `git annex init` will detect when it's run on a crippled filesystem and enable various workarounds, including [[direct_mode]].

The git-annex android app deals with all this automatically when creating a repository.
"""]]


code:
[[!comment format=mdwn
 username="Mebus"
 ip="2001:4dd0:ff00:29::2"
 subject="comment 2"
 date="2013-10-31T23:39:42Z"
 content="""
Hallo,

I tried it again.

On the Android tablet I get many files, which have got a size of 200 Bytes. And they should at least be a view kB.

Why?

Mebus

"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="209.250.56.47"
 subject="comment 3"
 date="2013-11-01T16:42:33Z"
 content="""
Files in the git repository whose content has not yet been downloaded will appear as these short placeholder files when using a filesystem that does not support symlinks. Once the files have been downloaded, that is replaced with their actual content. You may need to set up a way to your android tablet to download the files from the place they're being created.
"""]]


code:
[[!comment format=mdwn
 username="Mebus"
 ip="2001:4dd0:ff00:29::2"
 subject="comment 4"
 date="2013-11-11T20:24:11Z"
 content="""
Does it have to be the place, where they were created or can this also be a central server, I pushed everything to?

Mebus

"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="209.250.56.246"
 subject="comment 5"
 date="2013-11-13T16:37:46Z"
 content="""
The git-annex app will download the files from any remote you set up that has a copy of the file. A transfer remote in the cloud would work well.
"""]]


code:
I've got a Freebox Revolution set-top box / TV NAS at home, with some preconfigured layou for videos, photos and music.

I'd like to use it as a git-annex remote, but am afraid there's no support for FTP (authenticated) or SMB shares exported by the NAS (I don't think it supports other protocols even though runs Linux internally, AFAIK).

Is there any option to store much on that NAS and sync it with git-annex ?


code:
[[!comment format=mdwn
 username="http://cstork.org/"
 nickname="Chris Stork"
 subject="Similar to "direct special remotes""
 date="2013-08-16T17:57:26Z"
 content="""
You have a very similar problem as I do.

See my post where I called this [direct special remotes](http://git-annex.branchable.com/forum/Direct_special_remotes/).
"""]]


code:
[[!comment format=mdwn
 username="http://olivier.berger.myopenid.com/"
 nickname="obergix"
 subject="Except I don't have rsync/sftp on the NAS AFAICT"
 date="2013-08-17T08:56:37Z"
 content="""
There seemed to be some hope with rsync in your case, but not mine. Thanks anyway for the pointer.
"""]]


code:
[[!comment format=mdwn
 username="http://olivier.berger.myopenid.com/"
 nickname="obergix"
 subject="Experimenting with a CIFS mount point of the NAS and direct mode"
 date="2013-08-17T22:11:19Z"
 content="""
I've been experimenting with a direct mode repo on a CIFS mount of the SMB share of the NAS.

Unfortunately, it seems I can't propagate changes made on the laptop to the mount point by issueing a git pull or merge, as it is part of the unsafe commands that don't support direct mode (see details in <http://git-annex.branchable.com/direct_mode/#comment-fca93776b9526341e6aae4fe2c53038c>)... direct mode wouldn't be very useful then : I intend to mirror on the NAS the files I've been managing locally. For instance I will sort my photos in subdirs, on the laptop, and intend to mirror that on the NAS.
"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="4.154.0.63"
 subject="comment 4"
 date="2013-08-23T17:55:39Z"
 content="""
Just run `git annex sync` to update the direct mode repository on your NAS.
"""]]


code:
[[!comment format=mdwn
 username="http://olivier.berger.myopenid.com/"
 nickname="obergix"
 subject="So it seems I have found a sequence that seems to be operating fine"
 date="2013-08-23T20:28:45Z"
 content="""
Here's a script, which I think makes it work, using git clone, git annex copy and git annex sync :
    # The \"master\" remote which is with default indirect mode, on a Linux FS
    BASE1=~/tmp
    REPO1=$BASE1/annex-test

    # The \"slave\" remote on the NAS (a Samba server), which has been mounted with cifs, and thus will be in direct mode
    BASE2=/mnt/freebox-server/
    REPO2=$BASE2/annex-test

    cd $BASE1
    mkdir $REPO1
    cd $REPO1
    git init
    git annex init \"my laptop\"

    cd $REPO1
    cp -Lr ~/some_large_files ./
    git annex add some_large_files
    git commit -m \"added\"

    cd $BASE2
    git clone $REPO1 $REPO2
    cd $REPO2

    git annex init \"freebox server\"
    # This is not really needed, but if you want to replicate on a non cifs mount
    git annex direct

    cd $REPO1
    git remote add freebox-server $REPO2

    git annex copy --to freebox-server
    git annex sync

    cd $REPO2
    git remote add laptop $REPO1
    #git annex sync

After this, you should have your .git and plain \"direct\" files on the NAS, mirrored from what's on the laptop.

Hope this helps.
"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="4.154.0.63"
 subject="comment 6"
 date="2013-08-24T15:58:28Z"
 content="""
You can run any of these operations in any order and it will work. Ie, this is far more complicated and detailed than it needs to be. Just run git-annex commands in any order to do what you want to do. If they don't, file a (detailed) bug report.
"""]]


code:
I've imported about half of my photos into an annex on an external HDD,
using metadata extensively for EXIF info, including place names.

Checking out a new view is slower than I expected, at ~6 minutes.
Is this expected behavior, am I pushing the limits of file count already?

Is there anything I can do to speed things up?

    % ls -1 | wc -l
        7050

    % git branch -vv
      git-annex                      4e590d4 update
    * master                         985ba54 add jen's phone backups
      views/Year=_;Month=_;Address=_ 795a58b refs/heads/views/Year=_;Month=_;Address=_

    % /usr/bin/time -p git annex view "Year=*" "Address=*"
    view  (searching...)

    Checking out files: 100% (12789/12789), done.
    Switched to branch 'views/Year=_;Address=_'
    ok
    real       376.80
    user        17.98
    sys          9.66


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawmvzzyDA8uXFz8yokeCrepbh8PwWe_WrjE"
 nickname="Michael"
 subject="debug output"
 date="2014-06-16T14:17:06Z"
 content="""
So after looking for a debug flag, I see that it's spending all its time in cat-file, which I guess is probably not a surprise:

    % /usr/bin/time -p git annex view \"Year=*\" \"Address=*\" -d
    view  [2014-06-16 06:29:12 PDT] read: git [\"--git-dir=/Volumes/Four TB Backup/Photos/.git\",\"--work-tree=/Volumes/Four TB Backup/Photos\",\"symbolic-ref\",\"HEAD\"]
    [2014-06-16 06:29:12 PDT] read: git [\"--git-dir=/Volumes/Four TB Backup/Photos/.git\",\"--work-tree=/Volumes/Four TB Backup/Photos\",\"show-ref\",\"refs/heads/master\"]
    [2014-06-16 06:29:12 PDT] read: git [\"--git-dir=/Volumes/Four TB Backup/Photos/.git\",\"--work-tree=/Volumes/Four TB Backup/Photos\",\"symbolic-ref\",\"HEAD\"]
    [2014-06-16 06:29:12 PDT] read: git [\"--git-dir=/Volumes/Four TB Backup/Photos/.git\",\"--work-tree=/Volumes/Four TB Backup/Photos\",\"show-ref\",\"refs/heads/master\"]
    (searching...)
    [2014-06-16 06:29:12 PDT] read: git [\"--git-dir=/Volumes/Four TB Backup/Photos/.git\",\"--work-tree=/Volumes/Four TB Backup/Photos\",\"ls-files\",\"--cached\",\"-z\",\"--\",\"/Volumes/Four TB Backup/Photos\"]
    [2014-06-16 06:29:12 PDT] feed: git [\"--git-dir=/Volumes/Four TB Backup/Photos/.git\",\"--work-tree=/Volumes/Four TB Backup/Photos\",\"update-index\",\"-z\",\"--index-info\"]
    [2014-06-16 06:29:12 PDT] chat: git [\"--git-dir=/Volumes/Four TB Backup/Photos/.git\",\"--work-tree=/Volumes/Four TB Backup/Photos\",\"hash-object\",\"-w\",\"--stdin-paths\",\"--no-filters\"]
    [2014-06-16 06:29:12 PDT] read: git [\"--git-dir=/Volumes/Four TB Backup/Photos/.git\",\"--work-tree=/Volumes/Four TB Backup/Photos\",\"show-ref\",\"git-annex\"]
    [2014-06-16 06:29:12 PDT] read: git [\"--git-dir=/Volumes/Four TB Backup/Photos/.git\",\"--work-tree=/Volumes/Four TB Backup/Photos\",\"show-ref\",\"--hash\",\"refs/heads/git-annex\"]
    [2014-06-16 06:29:12 PDT] read: git [\"--git-dir=/Volumes/Four TB Backup/Photos/.git\",\"--work-tree=/Volumes/Four TB Backup/Photos\",\"log\",\"refs/heads/git-annex..4e590d433e01886e2823c3316d18b7e3cbafe227\",\"--oneline\",\"-n1\"]
    [2014-06-16 06:29:12 PDT] chat: git [\"--git-dir=/Volumes/Four TB Backup/Photos/.git\",\"--work-tree=/Volumes/Four TB Backup/Photos\",\"cat-file\",\"--batch\"]
    [2014-06-16 06:34:54 PDT] read: git [\"--git-dir=/Volumes/Four TB Backup/Photos/.git\",\"--work-tree=/Volumes/Four TB Backup/Photos\",\"write-tree\"]
    [2014-06-16 06:34:55 PDT] chat: git [\"--git-dir=/Volumes/Four TB Backup/Photos/.git\",\"--work-tree=/Volumes/Four TB Backup/Photos\",\"commit-tree\",\"c0fd1ea9ccae70a7353a2e787089b88115a42b7a\"]
    [2014-06-16 06:34:55 PDT] call: git [\"--git-dir=/Volumes/Four TB Backup/Photos/.git\",\"--work-tree=/Volumes/Four TB Backup/Photos\",\"update-ref\",\"refs/heads/views/Year=_;Address=_\",\"249c37d78e45d1fccfcb00f40337e05cb69fb64c\"]

    [2014-06-16 06:34:55 PDT] call: git [\"--git-dir=/Volumes/Four TB Backup/Photos/.git\",\"--work-tree=/Volumes/Four TB Backup/Photos\",\"checkout\",\"views/Year=_;Address=_\"]
    Checking out files: 100% (12789/12789), done.
    Switched to branch 'views/Year=_;Address=_'
    ok
    real       362.14
    user        18.16
    sys          7.70

"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="108.236.230.124"
 subject="comment 2"
 date="2014-06-16T17:52:34Z"
 content="""
View's don't scale to lots of files yet. Discussed in [[design/caching_database]]
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawmvzzyDA8uXFz8yokeCrepbh8PwWe_WrjE"
 nickname="Michael"
 subject="comment 3"
 date="2014-06-16T18:19:20Z"
 content="""
@joeyh, thanks - is that code all up to date here: https://github.com/joeyh/git-annex/tree/database ?

I'd be interested in helping test it.
"""]]


code:
I have configured git-annex on my Nexus 5. Even though it works fantastic, it shows some warning messages.

1: 'git-annex sync' :  WARNING: linker: git-annex has text relocations. This is wasting memory and is a security risk. Please fix.

2: 'git log': error: cannot run less. No such file or directory.

Both the commands will work has expected with these warnings. What could be the issue here.


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="209.250.56.146"
 subject="comment 1"
 date="2014-03-05T20:38:03Z"
 content="""
<http://stackoverflow.com/questions/20141538/mylib-so-has-text-relocations-this-is-wasting-memory-and-is-a-security-risk-pl>
Says that should have been fixed in ndk-r8c's gcc, but git-annex is already built with ndk-r9b. It might be though, that the llvm compiler still has the problem. In any case, I doubt there is a real security issue here, it's probably just a binary exploit hardening thing.

As for the git log, git users the less pager by default, but of course it's not necessary and so not included on Android. You can `git config core.pager cat` to get around this.
"""]]


code:
I'm trying to use git-annex as a dropbox-replacement, but I'm hitting unexpected things although I'm doing exactly what I see in the walkthrough screencast and/or the walkthrough commands. I'm using Ubuntu 13.10, installed git-annex with apt-get. It is version 4.20130815.

1) I launch the webapp and create an ~/annex folder as my initial repo. I copy some small text files in it, I see in webapp that they are added.
2) I add a new "Removable Storage" repo, in my usb drive. I do not touch any other settings, leave it as is. (ie in transfer group for example). It is successfully added to the repo list and I see sync complete and files synced on the webapp.
3) When I check the contents of the usb disk annex folder, I see the following, while I was expecting to see the files I synced instead.

emre@emrenb:~$ ls /media/emre/348B-78F0/annex/
annex  branches  config  description  HEAD  hooks  info  objects  refs

However, I expect the below which is the content of my ~/annex folder:
emre@emrenb:~$ ls annex/
unison-sync.log  unison-1.log  unison-2.log  unison.log

In the walkthrough video, when Joey adds a USB storage, the filenames & thir contents sync to the usb drive. Why doesn't it work for me?

Note that I tried also android app and also the zipped tarball build. Same result, I never see actual files.


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawk3HGoDpnOPob5jOjvIootmkve1-nCpRiI"
 nickname="Kalle"
 subject="Default removable storage repo not readable without git annex"
 date="2013-10-21T08:50:50Z"
 content="""
Hi,

Sounds to me you are expecting the files to be readable by a filemanager? By default a removable drive repo is only readable by git-annex.

The standard use case is to move files between computers with git-annex installed. I agree this is not what you´d expect.

Read <http://git-annex.branchable.com/forum/USB_backup_with_files_visible/> and the last post of <http://git-annex.branchable.com/forum/Can__39__t_get_git-annex_merge_to_work_from_git_hook/> for more info.
"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="64.134.31.139"
 subject="comment 2"
 date="2013-10-21T22:29:14Z"
 content="""
Kalle is right (I seem to keep saying that, Kalle!)

Following the walkthrough commands will get you a non-bare git repository on a removable drive, and you can run \"git annex sync\" in there periodically and see all the files in your repository on the removable drive. The webapp does not make that kind of repository though, and in the screencast I'm showing the normal behavior of the webapp.
"""]]


code: Hello Joey, I would be very much interested in a walkthrough for direct mode, as detailed as the one currently published. I see the comments in the current walkthrough on some differences to direct mode, but to me it is not obvious what best practices for git-annex use would be in direct mode, with and without the assistant. For a mix of Linux, OS X and Windows installations in the homes, it may also be interesting to see how to best set up the individual machines. Many thanks -


code:
Hi

I created a git annex repo in one of my drives. But now all my files have turned into symbolic links.
I need to remove the git repo and get my files back. How can I do that.

Some of the softwares am using will not follow symbolic links, so need put the files back properly as soon as  possible.

Please help.
Thank you in advance :)


code:
[[!comment format=mdwn
 username="http://ypid.wordpress.com/"
 ip="213.153.84.215"
 subject="Direct mode"
 date="2014-06-27T07:36:40Z"
 content="""
Hi

[[direct_mode/]] is your friend.
"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="108.236.230.124"
 subject="comment 2"
 date="2014-07-02T17:04:29Z"
 content="""
git annex uninit
"""]]


code:
Hello,

I'm not sure about the differences and interactions between watch / assistant / webapp / direct mode. I think I figured the following out, can someone confirm this, and perhaps a few words to the documentation / man page?

- git annex watch uses inotify to find new files, and runs git annex add on them (it does not do regular git add)
- git annex assistant does the same as watch, but also runs git annex sync for each new file (does it also enable direct mode?)
- git annex webapp does the same as assistant, and also starts a webapp (in my case it immediately started sending files to origin, without asking for confirmation, which was surprising, I guess this is because I have * annex.numcopies=2 set, and there was only one copy. Still I interpreted the documentation as if it would only show me an interface, not start doing things right away.)

Do these commands do anything else than what I described above?

best regards,
Tom


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="4.154.3.125"
 subject="comment 1"
 date="2013-02-07T18:12:17Z"
 content="""
Most of this is explained pretty well on the [[man page|git-annex]].

`git annex webapp` doesn't do anything on its own, other than start the assistant if it is not already running, and point your browser at its built-in webapp. Starting the assistant manually will always have the same effect as opening the webapp.

The assistant syncs files as configured by the preferred content settings of repositories. There's an interface to disable this syncing in the webapp, or change the setting to something that doesn't prefer all content be synced to a repository. This syncing of content is different from `git annex sync`, which only syncs git repository data. It's more like running `git annex copy --auto`

The assistant enables direct mode when a new repository is created (using the webapp). It does not enable direct mode when run in an existing repository.
"""]]


code: I am using the webapp to watch 3 repositories. When I run git annex webapp it starts watching one of the repos but unless I switch to other repos it does not start noticing changes after switching to each repo once it works as intended.


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="209.250.56.227"
 subject="comment 1"
 date="2013-12-27T19:02:28Z"
 content="""
Each separate repository needs to have a separate `git annex assistant` process handling it.

Normally, these would all be started when `git annex assistant --autostart` is run, typically when you log into a desktop environment.

So, on Linux, this is done via a .desktop file (or you can arrange for the command to be run as the right user at boot if not using a desktop); on OSX it's done via a .plist file. On Windows and Android, nothing handles the autostarting yet.
"""]]


code:
[[!comment format=mdwn
 username="https://me.yahoo.com/a/FHnTlSBo1eCGJRwueeKeB6.RCaPbGMPr5jxx8A--#ce0d8"
 nickname="Hamza"
 subject="comment 2"
 date="2013-12-27T22:40:06Z"
 content="""
I don't have a desktop env. just plain xmonad, so the correct sequence to start the webapp is,

    git annex assistant --autostart && git annex webapp

right?
"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="209.250.56.227"
 subject="comment 3"
 date="2013-12-29T17:59:53Z"
 content="""
In this case, I would probably put `git annex assistant --autostart` in `.xinitrc`, or even make a @reboot cron job that starts it. The webapp can then be opened with `git annex webapp` at any time. (The webapp is not a separate process and does not need to be started at login.)
"""]]


code:
Webapp on ARM?
==============

Since the webapp is apparently now available on Android (not tested yet, but I plan to do it soon ;)), I was wondering what was the status of the webapp on ARM. Does it build, does it work, and if it does would it be possible to enable it in the Debian package for the next release?

For the record, I'm using git-annex on my NAS (Synology DS413j). I'm using the Debian armel package and running it in an Arch Linux chroot (a very simple setup). It works really well, and I'm extremely satisfied with git-annex. Thanks a lot for all your work, Joey *et al.*!


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="4.154.1.10"
 subject="comment 1"
 date="2013-07-22T19:13:38Z"
 content="""
To build the WebApp on arm, you need a ghc that supports template haskell on arm. While I have heard persistent rumors of such a thing existing, I have yet to see it.

It's possible to build the WebApp on arm without that, but it's a complicated process, involving first building on x86 with the same versions of the haskell libraries that you have on the arm system, and then using the EvilSplicer to generate a source tree with the template haskell expanded, which can then be built on arm. This build process is not really suitable for a Debian package.
"""]]


code:
I am having some weird issues with git annex under OS X.  I am new to git annex, and am generally familiar with git.  I really like the concept of git-annex and am trying to lean how to fit it into my workflow, but Im running into problems.

I have a folder that I would like to keep in sync across multiple computers and thumb drives.  Right now this folder consists of other folders and PDF files.  After initializing git and git-annex, all of the files turn into symlinks, as expected.

Looking at this folder in OSX's finder, none of the symlinks have preview images unfortunately, as expected.  Also, when trying to open the file (or symlink, and in this case PDFs) in finder, it opens the console which outputs a bunch of crap and then nothing happens.  Right clicking on the symlink in finder and using Open with... the default application to open the symlink to the PDF is OSX's Preview.app, as expected, however for some reason this isn't being used when simple double clicking.  If you click on the Preview.app option under the right click menu, Preview.app launches, but never opens the file.  Selecting an alternative such as the Skim.app pdf viewer (if installed) successfully launches the application and opens the file.

My questions are:
1. What is the cause of not being able to launch the file in the correct app by simply double clicking it?  Do I have something mis-configued?  Is it a bug with git-annex?  Is git-annex simply not set up to work with finder yet?
2. How come preview can't open the file but skim can?  This might be directly related to the above question, but maybe not.
3. Is it possible to get file previews for the symlinks in your annex folder?  Are there plans to enable this kind of thing in the future, if even possible?

Details about my setup.  I am running on a 32-bit Core Duo Macbook pro from 2006, so enviably I have to run 10.6.8 and its the highest OS version I can reach right now.  All my CLI packages are installed and up to date with Mac Homebrew.  I am running git version 1.7.12.  I freshly installed haskell-platform: stable 2012.2.0.0 via homebrew.  From there I installed git-annex-3.20120825.


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="4.153.8.30"
 subject="comment 1"
 date="2012-09-09T17:22:58Z"
 content="""
It is unfortunate that certain badly-coded programs fall over on symlinks. In theory, any program that doesn't try to be too smart for its own good will see a symlink as just like the file it links to, and should behave the same. This is the theory that led me to use symlinks in git-annex. But some programs say \"oh, a symlink! Let's see what file that's pointing at, and behave differently!\" I don't understand the mindset that leads to this kind of program being written, unless it's a program like `cp`, that has very good reasons for (at your command) following symlinks or not.

I know absolutely nothing about Mac OSX, having never used it (except for in a ssh session while porting git-annex to it). But I have seen a similar problem with the FBReader ebook viewer. [Bug report](http://bugs.debian.org/685433)

I was able to work around that bug by migrating my files to the SHA256E backend. This ensures that the keys stored in git-annex have extensions like .pdf etc, similar to the symlinks that point to them. You might try migrating some of your files too, and see if it makes OSX behave better. Although I thought OSX didn't rely on file extensions to determine file type..

Anyway, the command is:

    git annex migrate --backend=SHA256E

I have been considering switching the default backend to SHA256E to avoid this type of problem. Your testing will be important grist for that decision.
"""]]


code:
[[!comment format=mdwn
 username="http://mike.magin.org/"
 nickname="mmagin"
 subject="comment 2"
 date="2012-09-11T15:27:58Z"
 content="""
I've been using git-annex mostly on Mac OS X, and it rapidly caused me to switch to SHA256E.  I'd certainly be in favor of it being the default.  Aside from [[bugs/fsck thinks file content is bad when it isn't]] (fixed by the time I encountered it), I have had no issues with it so far.
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawkurjhi0CRJvgm7QNaZDWS9hitBtavqIpc"
 nickname="Bret"
 subject="comment 3"
 date="2012-09-12T09:23:51Z"
 content="""
Sorry for the delayed response.  I am in the middle of an apartment move.

I tried changing my default PDF viewer to Skim to see if that improved anything.  Nada.  Same thing.  Finder opens the terminal and outputs a bunch of nonsense.

I tried `git annex migrate --backend=SHA256E` and it seemed to really improve things from finders point of view.  Still not icon previews, but you can launch every file by double clicking it and preview is able to follow the symlink.

Note, I am on OS X 10.6.  I cant upgrade to 10.7+ due to my laptops aging processor (10.7+ requires 64-bit intel chips) and thus can't test git-annex on a more modern OS version.  Someone should test it out on a newer version of OS X before chaining any defaults, however, it seems to really improve things for the setup I have.
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawmBUR4O9mofxVbpb8JV9mEbVfIYv670uJo"
 nickname="Justin"
 subject="comment 4"
 date="2012-09-12T20:32:52Z"
 content="""
I see a migration to SHA256E currently outputs:

migrate issue.txt (checksum...) (Recording state in git...)
ok

Is that verifying the existing checksum or re-computing it to re-name the file?  If the latter, couldn't the SHA256 to SHA256E migration just rename the file?

I would be worried about this process hiding data corruption.. If it isn't using the old checksum but instead re-computing a new one then it would be easy to miss the fact that a files checksum changed during this process.
"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="4.153.8.30"
 subject="comment 5"
 date="2012-09-14T04:20:27Z"
 content="""
Good catch, I've made it verify the old checksum first.

+/- E migrations are not currently optimised.
"""]]


code:
Hi,

How can I resolve the conflict when it occurs?

Suppose I have 2 branches (master, current), When I merge these branches or while doing cherry-pick, if I get conflict how can I resolve it?

Thank You


code:
[[!comment format=mdwn
 username="http://joey.kitenet.net/"
 nickname="joey"
 subject="comment 1"
 date="2012-04-23T14:29:03Z"
 content="""
You handle conflicts in annexed files the same as you would handle them in other binary files checked into git.

For example, you might choose to `git rm` or `git add` the file to resolve the conflict.

[[Previous_discussion|forum/A_really_stupid_question]]
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawlYu7QmD7wrbHWkoxuriaA9XcijM-g5vrQ"
 nickname="Royal"
 subject="Resolving conflict"
 date="2012-04-24T03:59:31Z"
 content="""
Hi,
Now I am able to resolve the conflict.
Thank you.
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawlYu7QmD7wrbHWkoxuriaA9XcijM-g5vrQ"
 nickname="Royal"
 subject="Resolving conflict"
 date="2012-04-23T15:49:30Z"
 content="""
Thanks for the reply.

I am executing the following commands.

git init main
cd main
git annex init main
echo a > a
git annex add a
git commit -m Initial
git annex unlock a
echo aa > a
git annex add a
git commit -m first
git annex unlock a
echo aaa > a
git annex add a
git commit -m second
git log
git cherry-pick <Hash of first commit>

--------------------
Error:

error: could not apply 2be8f38... first
hint: after resolving the conflicts, mark the corrected paths
hint: with 'git add <paths>' or 'git rm <paths>'
hint: and commit the result with 'git commit'

How can resolve the the above conflict.
If I see the content of the file I will get the content of second commit.
Is there any way I can get the content for first commit(Like in git we have 'theirs' option.)

Thank you.
"""]]


code: As of right now (2012-05-24 at 18:00 UTC), the [[Walkthrough]] page is basically empty. Its entire contents are "A walkthrough of the basic features of git-annex." No links (other than the autogenerated "what links to this page" list at the bottom) and no contents. Any idea what happened?


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawl9sYlePmv1xK-VvjBdN-5doOa_Xw-jH4U"
 nickname="Richard"
 subject="comment 1"
 date="2012-05-24T19:18:55Z"
 content="""
It seems the pages that are supposed to be inlined are not being found even though they are in `doc/walkthrough/`.
"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 subject="comment 2"
 date="2012-05-24T20:15:19Z"
 content="""
Broken last night during upgrade, fixed now, thanks for noticing.
"""]]


code: I want to add an existing repository using the assistant. Is that the way?


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="71.80.94.56"
 subject="comment 1"
 date="2014-02-07T19:03:30Z"
 content="""
Sure, you can use that menu item. If you try it out and enter in the path of a repository, you'll see that it offers to either let you combine the two repositories together, or leave them as two unconnected repositories with different files managed by the assistant.
"""]]


code: Since all the annexed (in indirect mode) files are symlinks to topdir/.git/annex/...   moving files among directories at different levels is not that straightforward since symlinks would get broken.  And since there is not 'annex mv' command -- what is the best way?  (unlock is not the resolution since it copies the file, which might be prohibitively large and inefficient)


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 nickname="joey"
 subject="comment 1"
 date="2013-06-12T16:42:54Z"
 content="""
You can move the symlinks however you like (git mv, or regular mv and git add).

To fix up broken symlinks, you can either run `git annex fix`, or just commit the move. The pre-commit hook fixes up links automatically.
"""]]


code: What is the difference between "local computer" and "remote server" in the assistant's repository configuration menu? How does git-annex treat the targets differently?


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="2001:4978:f:21a::2"
 subject="comment 1"
 date="2013-01-25T22:39:20Z"
 content="""
Local computer pairs with another computer on your LAN. You don't need an account on that computer, it can belong to someone else and has to be running the webapp as well for them to verify the pairing.

Remote server is for people with a ssh server out there, and does not require it be running the assistant, or even have git-annex installed. (Though it works better if at least git-annex is installed on it.)
"""]]


code: What do I do when I have set up git-annex to back up a folder to an encrypted remote on a vps, and my laptop was stolen? This recently happened, and luckally I had another backup, but what would I do in the case that I have 1 computer using git-annex, backung ip a folder to an encrypted ssh remote on a vps, and I loose the computer? How do I get the data back then?


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="2001:4978:f:21a::2"
 subject="comment 1"
 date="2013-07-26T03:57:06Z"
 content="""
I'm glad you had another backup. That is wise.

When you have a special remote, that does not store your git repository, but only the contents of files. So if you lose all copies of your git repositories, you have lost all record of the file name associated with a file. It's the same as if `fsck` rescued inodes to `lost+found`; you'd have to manually rename the files to new names.

Then there's the added complication of the encryption. This depends on how it was set up. If you set up the remote using a GPG key (and didn't lose the key too), you can manually use that key to decrypt the files. On the other hand, if it generated its own key and stored it in the git repo (default with the assistant), then when you lose all copies of the git repo, the key is gone, and your encrypted files are useless.
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawln3ckqKx0x_xDZMYwa9Q1bn4I06oWjkog"
 nickname="Michael"
 subject="comment 2"
 date="2013-07-26T15:04:31Z"
 content="""
I thought the key was checked into .git in any case, but encrypted in the case of GPG.  So .git gone => everything gone in any case.
"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="2001:4978:f:21a::2"
 subject="comment 3"
 date="2013-07-26T16:45:26Z"
 content="""
Michael, you're right. Thanks for fact checking my late night comment! ;)
"""]]


code:
On Android 4.4.4, I can install the most recent git annex. I want to join a git annex running on my linux server. Git annex on android seems to be able to contact remote, and it asks if I want to 'merge' with it. I hope that did not mean erase the remote.  Anyway, the sync says it is proceeding, but when it is done, I cannot find a folder with remote copies of files.

I figure either it worked, or it destroyed my remote.

After new install, I have folders /sdcard/annex and /sdcard/git-annex.home. I don't find a new folder with name of remote repo, don't understand why.

Help?


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawkpIIYg6Fl7OFsOHVPEchZYj68A3dk4lVg"
 nickname="Paul"
 subject="install log"
 date="2014-07-15T12:02:59Z"
 content="""
Installation starting to /data/data/ga.androidterm
ac90d4f57076a59cac3a8017b89ac9b5ab4e5021
installing busybox
installing git-annex
installing git-shell
installing git-upload-pack
installing git
installing gpg
installing rsync
installing ssh
installing ssh-keygen
linking ./libexec/git-core/git-show to git
linking ./libexec/git-core/git-cherry-pick to git
linking ./libexec/git-core/git-revert to git
linking ./libexec/git-core/git-rerere to git
linking ./libexec/git-core/git to git
linking ./libexec/git-core/git-cat-file to git
linking ./libexec/git-core/git-archive to git
linking ./libexec/git-core/git-apply to git
linking ./libexec/git-core/git-repack to git
linking ./libexec/git-core/git-replace to git
linking ./libexec/git-core/git-verify-pack to git
linking ./libexec/git-core/git-name-rev to git
linking ./libexec/git-core/git-grep to git
linking ./libexec/git-core/git-format-patch to git
linking ./libexec/git-core/git-log to git
linking ./libexec/git-core/git-read-tree to git
linking ./libexec/git-core/git-fast-export to git
linking ./libexec/git-core/git-mktag to git
linking ./libexec/git-core/git-cherry to git
linking ./libexec/git-core/git-stage to git
linking ./libexec/git-core/git-blame to git
linking ./libexec/git-core/git-help to git
linking ./libexec/git-core/git-tag to git
linking ./libexec/git-core/git-column to git
linking ./libexec/git-core/git-clone to git
linking ./libexec/git-core/git-pack-redundant to git
linking ./libexec/git-core/git-rev-list to git
linking ./libexec/git-core/git-ls-files to git
linking ./libexec/git-core/git-update-index to git
linking ./libexec/git-core/git-merge-tree to git
linking ./libexec/git-core/git-shortlog to git
linking ./libexec/git-core/git-pack-refs to git
linking ./libexec/git-core/git-fsck to git
linking ./libexec/git-core/git-mv to git
linking ./libexec/git-core/git-index-pack to git
linking ./libexec/git-core/git-upload-archive to git
linking ./libexec/git-core/git-unpack-file to git
linking ./libexec/git-core/git-merge-recursive to git
linking ./libexec/git-core/git-checkout to git
linking ./libexec/git-core/git-mailsplit to git
linking ./libexec/git-core/git-whatchanged to git
linking ./libexec/git-core/git-remote-fd to git
linking ./libexec/git-core/git-fsck-objects to git
linking ./libexec/git-core/git-diff to git
linking ./libexec/git-core/git-diff-tree to git
linking ./libexec/git-core/git-check-ref-format to git
linking ./libexec/git-core/git-status to git
linking ./libexec/git-core/git-annotate to git
linking ./libexec/git-core/git-bisect--helper to git
linking ./libexec/git-core/git-mktree to git
linking ./libexec/git-core/git-check-ignore to git
linking ./libexec/git-core/git-clean to git
linking ./libexec/git-core/git-merge to git
linking ./libexec/git-core/git-var to git
linking ./libexec/git-core/git-get-tar-commit-id to git
linking ./libexec/git-core/git-remote-ext to git
linking ./libexec/git-core/git-stripspace to git
linking ./libexec/git-core/git-send-pack to git
linking ./libexec/git-core/git-fetch to git
linking ./libexec/git-core/git-ls-remote to git
linking ./libexec/git-core/git-init to git
linking ./libexec/git-core/git-notes to git
linking ./libexec/git-core/git-commit to git
linking ./libexec/git-core/git-receive-pack to git
linking ./libexec/git-core/git-prune to git
linking ./libexec/git-core/git-update-server-info to git
linking ./libexec/git-core/git-gc to git
linking ./libexec/git-core/git-show-ref to git
linking ./libexec/git-core/git-merge-subtree to git
linking ./libexec/git-core/git-describe to git
linking ./libexec/git-core/git-branch to git
linking ./libexec/git-core/git-checkout-index to git
linking ./libexec/git-core/git-prune-packed to git
linking ./libexec/git-core/git-ls-tree to git
linking ./libexec/git-core/git-hash-object to git
linking ./libexec/git-core/git-push to git
linking ./libexec/git-core/git-for-each-ref to git
linking ./libexec/git-core/git-verify-tag to git
linking ./libexec/git-core/git-diff-files to git
linking ./libexec/git-core/git-check-attr to git
linking ./libexec/git-core/git-mailinfo to git
linking ./libexec/git-core/git-check-mailmap to git
linking ./libexec/git-core/git-fetch-pack to git
linking ./libexec/git-core/git-diff-index to git
linking ./libexec/git-core/git-add to git
linking ./libexec/git-core/git-config to git
linking ./libexec/git-core/git-merge-ours to git
linking ./libexec/git-core/git-symbolic-ref to git
linking ./libexec/git-core/git-init-db to git
linking ./libexec/git-core/git-pack-objects to git
linking ./libexec/git-core/git-rm to git
linking ./libexec/git-core/git-write-tree to git
linking ./libexec/git-core/git-show-branch to git
linking ./libexec/git-core/git-fmt-merge-msg to git
linking ./libexec/git-core/git-count-objects to git
linking ./libexec/git-core/git-reset to git
linking ./libexec/git-core/git-reflog to git
linking ./libexec/git-core/git-rev-parse to git
linking ./libexec/git-core/git-update-ref to git
linking ./libexec/git-core/git-patch-id to git
linking ./libexec/git-core/git-commit-tree to git
linking ./libexec/git-core/git-merge-file to git
linking ./libexec/git-core/git-credential to git
linking ./libexec/git-core/git-unpack-objects to git
linking ./libexec/git-core/git-bundle to git
linking ./libexec/git-core/git-merge-index to git
linking ./libexec/git-core/git-remote to git
linking ./libexec/git-core/git-merge-base to git
linking ./bin/git-upload-archive to git
linking ./bin/git-receive-pack to git
linking ./libexec/git-core/git-shell to git-shell
linking ./libexec/git-core/git-upload-pack to git-upload-pack
Installation complete

"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="209.250.56.2"
 subject="comment 2"
 date="2014-07-15T18:37:34Z"
 content="""
It'll be in `/sdcard/annex` by defualt, unless you told the webapp to put it somewhere else when it asked where to put the repository. `/sdcard/annex` is a git repository which should contain all the files from the remote (not in any separate subdirectory, that's not how git works). If it does not, you can run commands like `git log --stat` in that repository to see what files have been added/removed, or `git annex status` to check the status of the files.
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawkpIIYg6Fl7OFsOHVPEchZYj68A3dk4lVg"
 nickname="Paul"
 subject="text relocations refusal"
 date="2014-07-16T02:19:27Z"
 content="""
I'm afraid I've ruined the remote repo. For directory, I wrote \"mediashare\" because that's where I want copy to go.  i don't want it in default \"annex\". In git assistant, it is unclearbwhst directory means.   Mount point for cooy of remote?

Well, this is all I get now...

1|u0_a127@manta:/sdcard $ cd annex
u0_a127@manta:/sdcard/annex $ git annex status
WARNING: linker: git-annex has text relocations. This is wasting memory and is a security risk. Please fix.

"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="209.250.56.2"
 subject="comment 4"
 date="2014-07-16T20:41:17Z"
 content="""
That warning about text relocations always happens on Android and is not relevant to whatever problem you're having.

If git-annex status does not print anything else, then it seems you have a git-annex repository in /sdcard/annex. If you entered \"mediashare\" in the webapp, you might have another repository in another location. When you open the webapp, it tells you the location of the repository in the upper-right corner -- \"Repository: $location\"
"""]]


code:
So Box.net and Amazon is supported, any way to get google drive in there?

thanks for cool software!


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="4.154.0.140"
 subject="comment 1"
 date="2013-07-18T17:40:36Z"
 content="""
[[tips/googledriveannex]]

See [[special_remotes]] for a full list of things people have gotten to work.
"""]]


code:
I currently have a pair of hard drives that I store backups etc on and that I would like to use as archive repositories for Git-Annex. I want everything on these to be encrypted, because I can't guarantee physical security of them.

When I am setting up a removable drive as a repository, there is no option to encrypt to it. I understand that there is a "Directory" special remote, but it seems odd to have to use a different special remote that is not built into the webapp. Wouldn't it be good to have a checkbox for encryption when setting up a removable drive as a repository?

Is there something that I'm missing here?

Thanks for a great piece of software!


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="4.154.4.90"
 subject="comment 1"
 date="2013-07-16T19:45:42Z"
 content="""
git-annex stores a clone of the git repository on the removable drive. To encrypt a git remote, you can use the [git-remote-gcrypt](https://github.com/blake2-ppc/git-remote-gcrypt) tool. This will work with git-annex, but it needs integration into the webapp to support setting it up, and further integration so git-annex can detect when a git remote is encrypted and also encrypt the contents of file is stores there.
"""]]


code:
I created a test remote using the command:

    git annex initremote mybup type=bup encryption=none buprepo=/tmp/mybup

I can copy files to and from the remote just fine.  However, every time I do so it makes changes inside `~/.bup`.  If I delete `~/.bup` it will recreate it.  Is this expected?  Are the files in `~/.bup` of any consequence?


code:
[[!comment format=mdwn
 username="https://me.yahoo.com/a/2grhJvAC049fJnvALDXek.6MRZMTlg--#eec89"
 nickname="John"
 subject="Ah"
 date="2012-10-18T23:52:03Z"
 content="""
I get now why ~/.bup is being used: -r DIR specifies a local DIR as the bup directory, but ~/.bup is still used for the local temp files.  It would be nice, therefore, if BUP_DIR were set in the case that -r specifies a local directory, so that only that directory is used.  Unless, of course, that leads to space limitations in the bup directory.  Hmm... maybe this should just be left as is in that case.
"""]]


code:
Here's my current situation:

I have a box which creates about a dozen files periodically. All files add up to about 1GB in size. The files are text and sorted. I then rsync the files to n servers. The rsync diff algorithm transfers way less than n * 1GB because the files are largely the same. However, this distribution technique is inefficient because I must run n rsync processes in parallel and the rsync diff algorithm takes a lot of CPU.

How could I use git-annex instead of rsync?

Because the box producing the new files also has the old files, then presumably git could calculate the diffs for each file once instead of n times as with the rsync solution? Then only the diffs need be distributed to the n servers... using git-annex? And finally the newly updated version of the dozen files needs to be available on each of the n servers. Ideally, the diffs would not mount up over time on either the publishing server or the n servers, thus causing out of disk problems etc. How to deploy git-annex to solve my problem?


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 nickname="joey"
 subject="comment 1"
 date="2013-03-27T22:48:16Z"
 content="""
git-annex will not perform better than rsync. It uses rsync. It also does not yet use old versions of files as a rsync basis when transferring a new version of a file, so is not as efficient in this scenario.
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawkaBh9VNJ-RZ26wJZ4BEhMN1IlPT-DK6JA"
 nickname="Alex"
 subject="Bup"
 date="2013-04-03T05:47:41Z"
 content="""
I think bup will solve your use case. Make a local bup repo and backup there then clone that repo n times. If you run git annex on top of bup you can then retrieve the subset of files you need at each destination. This will work best if you don't need every file on every remote or if your network is much slower than local disk access.
"""]]


code:
FAT32 does not support symlinks, so I wonder if there's going to be a problem with that.

Generally speaking, I am wondering about portability of git annex on windows and on android...


code:
[[!comment format=mdwn
 username="http://joey.kitenet.net/"
 nickname="joey"
 subject="comment 1"
 date="2011-03-07T19:13:14Z"
 content="""
See [[bugs/fat_support]]. A bare git repo will have to be used to avoid symlink problems, at least for now. The other problem is that git-annex key files have colons in their filenames.
"""]]


code:
[[!comment format=mdwn
 username="http://joey.kitenet.net/"
 nickname="joey"
 subject="comment 2"
 date="2011-03-19T15:37:22Z"
 content="""
Now it's fully supported, so long as you put a bare git repo on your key.
"""]]


code:
I am trying to connect to dreamhost's objects (S3) platform.  I am using the windows version.
Am I specifying the host correctly? Any help is appreciated.

$ git-annex initremote host="objects.dreamhost.com" cloud keyid=XXXXX type=S3

initremote host=objects.dreamhost.com (encryption setup) (hybrid cipher with gpg
 key 70827ADCDE25DA0F) (checking bucket...)
git-annex: user error (openTCPConnection: host lookup failure for "host=objects.
dreamhost.com-f5573b49-3668-4f94-a1da-aa55085c45e8.s3.amazonaws.com")
failed
git-annex.exe: initremote: 1 failed


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="209.250.56.87"
 subject="comment 1"
 date="2013-12-12T17:38:21Z"
 content="""
No, you cannot quote part of a parameter like that. git-annex thinks you are trying to use a host that includes the quote marks you put around it.

Since you're using Windows, I can't give any advice on how to use quoting at the shell/cmd prompt. However, I don't see any need for quoting here anyway.
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawk_bIB1Vadfc_GfPLKX_ScfWFCcaGyk8Ac"
 nickname="Rob"
 subject="Remove quotes"
 date="2013-12-13T04:44:22Z"
 content="""
I tried the command with and without quotes it results in the same error.  I think the issue is that the host is trying to resolve a concatenation of the hostname and amazon.com when in fact it should be resolving to the host name only.
"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="209.250.56.87"
 subject="comment 3"
 date="2013-12-16T17:07:17Z"
 content="""
Looking at this again, you forgot to tell it the name of the remote, so it took the first parameter (host=\"objects.dreamhost.com\" ) as the name. The rest probably follows from that mistake.

use `git annex initremote mys3 host=...`
"""]]


code:
Hi,

Do you have any news about Windows support?
Is this something you're currently working on?

Thanks!


code:
[[!comment format=mdwn
 username="http://joey.kitenet.net/"
 nickname="joey"
 subject="comment 1"
 date="2012-03-12T06:43:02Z"
 content="""
[[todo/windows_support]] has everything I know about making a windows port. This badly needs someone who understand Windows to dive into it. The question of how to create a symbolic link (or the relevant Windows equivilant) from haskell on Windows
is a good starting point..
"""]]


code:
Having a spot of bother in setting up for windows usage.

I'm attempting to have a windows box syncing to a server (over ssh) and a linux box also syncing against that*

So, on each machine I do

 git init
 git annex init

On the windows and linux desktops I then do a

 git remote add server serverdetails.

Now the problem is that if I don't add files to the repos on the machines, they won't sync as there is no branch checked out; and if I do then the first one is fine but the second will fail as it doesn't allow fast-forwards. What am I doing wrong? I've tried making the server repo bare / not bare.

I'm using the latest nightly windows build, and a build from git from today (29d5bb94b4512cfe3072c9ff840cb0ce9f2af744)

*Actually I'm trying to do something a little more complex than that, but this is the simplest version I can come up with.


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawmhfodZquCI_EEl-f3h7HkROTszlsQL6yA"
 nickname="Joe"
 subject="comment 1"
 date="2013-06-16T11:29:36Z"
 content="""
This may be related to [this](http://git-annex.branchable.com/bugs/windows_port_-_repo_can__39__t_pull_newly_added_files_/#comment-a023dca06c2bcb9636d8a332f8e95ae4)
"""]]


code:
**EDIT: Mistakenly posted this thread in the forum.  I created a new post in [[todo|todo/Bittorrent-like_features]]

Do you think it would be possible to have bittorrent-like transfers between remotes, so that no one remote gets pegged too hard with transfers? It would be great if you distribute your files between multiple bandwidth-capped remotes, and want fast down speed.  Obviously, this isn't a simple task, but the protocol is already there, it just needs to be adapted for the purpose (and re-written in Haskell...).  Maybe some day in the future after the more important stuff gets taken care of?  It could be an enticing stretch goal.

PS: still working on getting BTC, will be donating soon!


code:
[[!comment format=mdwn
 username="http://a-or-b.myopenid.com/"
 ip="203.45.2.230"
 subject="comment 1"
 date="2013-07-19T02:33:36Z"
 content="""
I agree.  I mentioned it briefly here:

[[http://git-annex.branchable.com/design/assistant/polls/what_is_preventing_me_from_using_git-annex_assistant/#comment-5a5d46967b826f602c423d7f72ac6f5e]]

I did *briefly* look into torrents and the like, but couldn't work out a way to lock it down so only authorized users could use either the tracker or connect to pull the data down...

The security aspect is the kicker in my mind.
"""]]


code:
[[!comment format=mdwn
 username="GLITTAH"
 ip="37.130.227.133"
 subject="comment 2"
 date="2013-07-19T07:38:47Z"
 content="""
Okay I've got some ideas, but they need refining.  I should be posting more tomorrow.

One thing I thought of though, there could be two types of torrent support:

1.  You use a torrent file or magnet link as a special remote URL.  This remote is read-only, naturally, and you'd only be able to leech (and seed, I suppose) the file in the torrent.  Neat feature maybe.  I wasn't previously thinking of this.

2. This isn't really bittorrent, just inspired by it.  You use a bittorrent-type protocol to transfer between your private remotes.  You could download from multiple remotes at once to get a file (like leeching, without a tracker, just pre-defined peers...aka remotes), or you could superseed to remotes to push a file, and have the remotes then share the parts of the file amongst themselves to each get a complete copy.  Remote groups could possibly be built on, or special groups could be coded in to define the swarm.  You update (add/remove) the swarm members locally, and push the changes to each of the old swarm members so they can add/remove other remotes (or themselves) from the swarm.  Possibly, you could just push the updated swarm list to one remote already in the swarm, it adopts the new list, and shares it with the rest of the swarm.  Each remote does the same.  Maybe the swarm list should be GPG-signed and verified before adopted and passed forward?

The more I think about it, the more awesome I think it could be, but holy shit it would be a LOT of work.
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawmLB39PC89rfGaA8SwrsnB6tbumezj-aC0"
 nickname="Tobias"
 subject="comment 3"
 date="2013-07-19T07:50:09Z"
 content="""
Personally i think the easiest implementation to make would be smart transfers.

You have a repo on your laptop, and a repo at work. And you have(say) 5 special remotes in between them that they need to use to transfer data.

The laptop could upload 5 different files to those repositories. Wouldn't quite be bittorent transfer(not even close). But it would probably be much simpler to implement. And when syncing a lot of files(instead of just one really big one) the speed gain should really be about the same as if all the special remotes used some kind of bittorent thing.

Just a thought.
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawmLB39PC89rfGaA8SwrsnB6tbumezj-aC0"
 nickname="Tobias"
 subject="comment 4"
 date="2013-07-19T07:51:39Z"
 content="""
Of course all files should be uploaded to all remote(to match with numcopies). but the initial push could be spread out over all existing remotes, to facility faster initial download/transfer.
"""]]


code:
[[!comment format=mdwn
 username="GLITTAH"
 ip="37.130.227.133"
 subject="comment 5"
 date="2013-07-19T21:23:35Z"
 content="""
Disclaimer: I'm thinking out loud of what could make git-annex even more awesome.  I don't expect this to be implemented any time soon.  Please pardon any dumbassery.

Much easier to implement, but having your remotes (optionally!) act like a swarm would be an awesome feature to have because you bring in a lot of new features that optimize storage, bandwidth, and overall traffic usage.  This would be made a lot easier if parts of it were implemented in small steps that added a nifty feature.  The best part is, each of these could be implemented by themselves, and they're all features that would be really useful.

Step 1. Concurrent downloads of a file from remotes.

This would make sense to have, it saves upload traffic on your remotes, and you also get faster DL speeds on the receiving end.

Step 2. Implementing part of the super-seeding capabilities.

You upload pieces of a file to different remotes from your laptop, and on your desktop you can download all those pieces and put them together again to get a complete file.  If you *really* wanted to get fancy, you could build in redundancy (ala RAID) so if a remote or two gets lost, you don't lose the entire file.  This would be a very efficient use of storage if you have a bunch of free cloud storage accounts (~1GB each) and some big files you want to back up.

Step 3. Setting it up so that those remotes could talk to one another and share those pieces.

This is where it gets more like bittorrent.  Useful because you upload one copy and in a few hours, have say, 5 complete copies spread across your remotes.  You could add or remove remotes from a swarm locally, and push those changes to those remotes, which then adapt themselves to suit the new rules and share those with other remotes in the swarm (rules should be GPG-signed as a safety precaution).  Also, if/when deltas get implemented, you could push that delta to the swarm and have all the remotes adopt it.  This is cooler than regular bittorrent because the shared file can be updated.  As a safety precaution, the delta could be GPG signed so a corrupt file doesn't contaminate the entire swarm.  Each remote could have bandwidth/storage limits set in a dotfile.

This is a high-level idea of how it might work, and it's also a HUGE set of features to add, but if implemented, you'd be saving a ton of resources, adding new use cases, and making git-annex more flexible.

"""]]


code:
[[!comment format=mdwn
 username="GLITTAH"
 ip="37.130.227.133"
 subject="comment 6"
 date="2013-07-19T21:28:58Z"
 content="""
Obviously, Step 3 would only work on remotes that you have control of processes on, but if given login credentials to cloud storage remotes (potentially dangerous!) they could read/write to something like dropbox or rsync.

Another thing, this would be completely trackerless.  You just use remote groups (or create swarm definitions) and share those with your remotes.  It's completely decentralized!
"""]]


code:
Maybe this explained somewhere else, but what is the purpose of marking files readonly. To me this is an annoyance that doesn't make sense. I want to be able to change my files, having to go through an unlock step, to do that seems like unnecessary work. Interestingly, Microsoft's Team Foundation Server source control does the same thing and I don't like it there either.

In addition why replace files with symlinks? Why not just leave the real files in place, or do the reverse and put the symlink to the file in the repository.


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawmL8pteP2jbYJUn1M3CbeLDvz2SWAA1wtg"
 nickname="Kristian"
 subject="comment 1"
 date="2012-11-22T23:38:33Z"
 content="""
By using symlinks gits problems with big files goes away. You simply can't put 600 GB in a git repo - that's the way I use git-annex. Also having files read only by default eliminates the need to store two copies of all files in one form or another.

"""]]


code:
I have a DLink Boxee media player and it can not play content from symbolic links, it needs to access regular media files. Unfortunately unlocking/locking is quite slow for such a large amount of data due to the required data copying, but it should not even be needed since I do not need write access to any file to watch the movie or to play the song.

Is it currently possible or would it be possible to add a commands like "unlock" which would not copy the file data but simply move files out from the data store into the tree while still keeping the files read only? A corresponding "lock" command would also be needed to restore the normal symbolic link tree structure.

Update:
I tried the rsync special remote http://git-annex.branchable.com/special_remotes/rsync/ and it works but the file structure created reflects the data store not the view given by the symbolic links.


code:
[[!comment format=mdwn
 username="http://joey.kitenet.net/"
 nickname="joey"
 subject="comment 1"
 date="2011-07-07T15:27:28Z"
 content="""
The rsync or directory special remotes would work if the media player uses metadata in the files, rather than directory locations.

Beyond that there is the [[todo/smudge]] idea, which is hoped to be supported sometime.
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawmL8pteP2jbYJUn1M3CbeLDvz2SWAA1wtg"
 nickname="Kristian"
 subject="Solution"
 date="2011-07-31T15:24:25Z"
 content="""
Yes, it can read id3-tags and guess titles from movie filenames but it sometimes gets confused by the filename metadata provided by the WORM-backend.

I think I have a good enough solution to this problem. It's not efficient when it comes to renames but handles adding and deletion just fine

    rsync -vaL --delete source dest

The -L flag looks at symbolic links and copies the actual data they are pointing to. Of course \"source\" must have all data locally for this to work.

"""]]


code:
It would be extremely useful to have some additional ways to select files (for git annex copy/move/get and maybe others) based on the meta-information available to git-annex, rather than just by file or directory name.

An example of what I'd like to do is this:

    host1$ git annex copy --to usb-drive --missing-on host2

This would check location tracking information and copy each file from host1's annex which is not present on host2 onto the usb-drive annex -- i.e. it's what I want when I need to do a sneakernet synchronisation of host1 and host2 (for backup purposes, for example). Note that of course I could copy --to host2, assuming network connectivity, but that would take a long time.

There's probably other selectors that we can imagine; an obvious one could be --present-on <annex> -- useful for judiciously dropping only those files that you have easily available in a local annex (as you may want to keep files that are hard to make available even if --numcopies would nominally be satisfied).

Other similar ideas for file content selectors:

 * Files that have less than n, exactly n or more than n copies -- for when you need to satisfy your --numcopies policy over sneakernet.
 * Files that are present (or not present) on some trusted annex -- for making sure you have trusted copies of everything.
 * Boolean combinations of these filters -- "git annex drop --present-on lanserver1 --or --present-on lanserver2" or similar syntax, although obviously doing this in full generality may be quite fiddly.


code:
[[!comment format=mdwn
 username="http://christian.amsuess.com/chrysn"
 nickname="chrysn"
 subject="filtering based on git-commits"
 date="2011-06-23T13:56:35Z"
 content="""
additional filter criteria could come from the git history:

* `git annex get --touched-in HEAD~5..` to fetch what has recently been worked on
* `git annex get --touched-by chrysn --touched-in version-1.0..HEAD` to fetch what i've been workin on recently (based on regexp or substring match in author; git experts could probably craft much more meaningful expressions)

these options could also apply to `git annex find` -- actually, looking at the normal file system tools for such tasks, that might even be sufficient (think `git annex find --numcopies-gt 3 --present-on lanserver1 --drop` like `find -iname '*foo*' -delete`

(i was about to open a new forum discussion for commit-based getting, but this is close enough to be usefully joint in a discussion)
"""]]


code:
[[!comment format=mdwn
 username="http://joey.kitenet.net/"
 nickname="joey"
 subject="comment 2"
 date="2011-09-19T18:46:35Z"
 content="""
This is now almost completely implemented. See [[walkthrough/powerful_file_matching]].

"""]]


code:
I think it would be useful to supplement the `reinject` command with an automatic
mode which calculates the checksum of the source file and injects the file if it
is known to the repository (without the need to provide a destination filename).
In addition, this could be done recursively if the user provides a directory to
inject. All this can probably be done already with some plumbing, but a simple
`reinject --auto` (or `scour`, or `scavenge`, if you like) would be a nice addition.
Of course this would only work for the checksum backends.

Example use cases would be:

* Recovering data from lost+found easily
* Making use of old (pre-git-annex) archival volumes with useful files
  scattered among non-useful files
* Sneaker-netting files between disconnected git-annex repositories


code:
I'm not sure if this _feature_ exists already wrapped or provided as a recipe for users or not yet. But it would be nice to be able to do a

    git annex du [PATH]

Such that the output that git annex would return is the total disk used locally in the PATH and the theoretical disk used by the PATH if it was fully populated locally. e.g.

    $ git annex du FSL0001_ANALYSIS
    $ Local: 1000kb, Annex: 2000kb

or something along the lines of that?


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="4.153.2.25"
 subject="comment 1"
 date="2012-06-27T12:36:08Z"
 content="""
Use `du -L` for the disk space used locally. The other number is not currently available, but it would be nice to have. I also sometimes would like to have data on which backends are used how much, so making this `git annex status --subdir` is tempting. Unfortunatly, it's current implementation scans `.git/annex/objects`
and not the disk tree (better for accurate numbers due to copies), so it would not be a very easy thing to add. Not massively hard, but not something I can pound out before I start work today..
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawmBUR4O9mofxVbpb8JV9mEbVfIYv670uJo"
 nickname="Justin"
 subject="comment 2"
 date="2012-06-27T12:45:42Z"
 content="""
I have a hacked up version of sharebox that does this.. I need to fix it up and push it to github..

the short of it is that you can do

    def calculate_size(path):
        annexfile = os.path.basename(os.readlink(path))
        #SHA256-s2007550713--....
        size = annexfile.split(\"-\")[1]
        return int(size[1:])

to get the size of files.. a 'git-annex du' should be pretty straightforward...
"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 nickname="joey"
 subject="comment 3"
 date="2013-03-11T05:27:38Z"
 content="""
You can now use \"git annex status .\"

Example:

<pre>
git annex status roms
directory: roms
local annex keys: 1
local annex size: 248 megabytes
known annex keys: 277
known annex size: 12 gigabytes
</pre>
"""]]


code: Logging to file would be nice when running git-annex as a daemon so when something fails or when certain events happens one can look at the logs to see what has failed, unless I'm missing something this probably should be on the wishlist or roadmap.


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="4.153.14.60"
 subject="comment 1"
 date="2012-06-23T14:30:22Z"
 content="""
The logging format could be improved, but the daemon already logs to .git/annex/daemon.log. It also automatically rotates the log file.
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawkSq2FDpK2n66QRUxtqqdbyDuwgbQmUWus"
 nickname="Jimmy"
 subject="comment 1"
 date="2012-06-23T08:00:12Z"
 content="""
actually, scratch that, i found it. it was in _.git/annex/daemon.log_ along with the other bits and pieces
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawkSq2FDpK2n66QRUxtqqdbyDuwgbQmUWus"
 nickname="Jimmy"
 subject="comment 2"
 date="2012-07-03T14:48:16Z"
 content="""
Adding a date and timestamp would be a nice start to improving things.
"""]]


code:
I have several remotes which are not always accessible. For example they can
be on hosts only accessible by LAN or on a portable hard drive which is not
plugged in. When running sync these remotes are checked as well, leading to
unnecessary error messages and possibly git-annex waiting for a few minutes
on each remote for a timeout.

In this situation it would be useful to mark some remotes as offline
(`git annex offline <remotename>`), so that git-annex would not even attempt
to contact them. Then, I could configure my system to automatically, for example,
mark a portable hard disk remote online when plugging it in, and offline when
unplugging it, and similarly marking remotes offline and online depending on
whether I have an internet connection or a connection to a specific network.


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 subject="comment 1"
 date="2012-07-06T13:04:07Z"
 content="""
You can already do this:

    git config remote.foo.annex-ignore true

There's no need to do anything for portable drives that are sometimes mounted and sometimes not -- git-annex will automatically avoid using repositories in directories that do not currently exist.

I thought git-annex also had a way to run a command and use its exit status to control whether a repo was
ignored or not, but it seems I never actually implemented that. It might be worth adding, although the command would necessarily run whenever git-annex is transferring data around.
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawncBlzaDI248OZGjKQMXrLVQIx4XrZrzFo"
 nickname="Perttu"
 subject="comment 2"
 date="2012-07-07T17:45:43Z"
 content="""
Ah, I didn't read the man page carefully enough. My apologies.

Setting the ignore status based on an exit status would be
even better, since this avoids re-writing a new config file for
each repository each time I enter or exit my LAN.
"""]]


code:
Since _transfer queueing_  and syncing of data works now in the assistant branch (been playing with it), there are times when I really don't want to sync the data, I would like to just sync meta-data and manually do a _get_ on files that I would want or selectively sync data in a subtree.

It would be nice to have the syncing/watch feature to have the option of syncing only *meta-data* or *meta-data and data*, I think this sort of option was already planned? It would also be nice to be able to automatically sync data for only a subtree.

My use case is, I have a big stash of files somewhere at home or work, and I want to keep what I am actually using on my laptop and be able to selectively just take a subtree or a set of subtree's of files. I would not always want to suck down all the data but still have the functionally to add files and push them upstream and sync meta-data.

that is...

> * Site A: big master annex in a server room with lots of disk (or machines), watches a directory and syncs both data and meta-data, it should always try and pull data from all it's child repos. That way I will always have a master copy of my data somewhere, it would be even nicer if I could have clones of the annex, where each annex is on a different machine which is configured to only sync a subtree of files so I can distribute my annex across different systems and disks.
>   * Site A: machine A: syncs Folder A
>   * Site A: machine B: syncs Folder B
>   * and so on with selectively syncing sites and directories
> * Laptop: has a clone of the annex, and watches a directory, syncs meta-data as usual and only uploads files to a remote (all or a designated one) but it never downloads files automatically or it should only occur inside a selected subtree.


code:
Hi Joey,
    I recently tried cloning a music annex to a microSD card to play on rockbox (FAT only) with my Sansa Clip, but 'git clone' stops the first time it runs into something with a special character in a file or directory, like a colon in an album title.  After this happens, there doesn't seem to be a way to get a full annex file/directory listing for the annex; it only has up to where git failed since FAT can't handle the name.  Is it possible for git-annex to replace special characters in directory/filenames with a space or underscore if it's on a crippled filesystem?  Also, would it be possible to then clone from that annex to a non-crippled FS, preserving the original filenames with special characters?  I've been meaning to ask about this for a while, so I'm a teeny bit fuzzy on the specifics...if you want, I'll reproduce the problem and cough up some more details.

PS: I love git-annex!


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 nickname="joey"
 subject="comment 1"
 date="2013-04-06T21:13:44Z"
 content="""
I think this is a general git question, not specific to git-annex. At the point in time that you run `git clone` on the FAT filesystem, git-annex is not running at all.

You could ask the git community about this. I know that git already has a core.precomposeunicode setting that is used to work around a somewhat similar problem on Mac OS.

What git-annex *does* let you do is set up a [[directory_special_remote|special_remote/directory]] on the FAT filesystem. This way,
there is no git repository there, and the filenames stored in git are not used at all on FAT. If your media player looks at mp3 metadata to get song titles, etc and doesn't care about the filenames at all, this would be a good option.
"""]]


code:
What is the correct way to add/modify files such that the changes are synchronized?

Suppose I have a server "server" and a windows client "client" (more after some time); everything direct mode.

I created the repos on the server:

     cd bin
     git init
     git annex init server
     git annex direct
     git annex add .
     git annex sync

On the (Windows) client using cygwin:

     git annex clone ssh://me@server:/srv/bin
     cd bin
     git annex init client

Then I want to add files on the client:

     git annex add tools
     git annex sync

At this point I wonder that the data is not copied to the server but only metadata!
I then used

     git annex sync --content

but then everything from the server is leeched as well (as if I could have called "git annex get .")

What is the intented workflow such that added/modified files on the client always go to the server, modified/new files from the server are always pulled but ONLY if I previously got the file/directory via "get annex get"?


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawnJO6OKamxo1HDLwdV-W3KV4GhJ5Qitl9M"
 nickname="Frederik"
 subject="comment 1"
 date="2014-05-15T00:57:23Z"
 content="""
Check the description for manual out on [[http://git-annex.branchable.com/preferred_content/standard_groups/]]. I believe that's what you'll need.
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawk9SYh6N-JUMkYkW4aOk55zC3Vr9KonDV4"
 nickname="Florian"
 subject="comment 2"
 date="2014-06-18T07:35:31Z"
 content="""
Just to be sure I understood the topic, this would be achieved in putting the repo in the manual transfer group?

manual

This gives you nearly full manual control over what content is stored in the repository. This allows using the assistant without it trying to keep a local copy of every file. Instead, you can manually run git annex get, git annex drop, etc to manage content. Only content that is already present is wanted.

The exception to this manual control is that content that a client repository would not want is not wanted. So, files in archive directories are not wanted once their content has reached an archive repository.

present and ($client)

(Where $client is a copy of the preferred content expression used for clients.)
<<<
"""]]


code:
Hi,

this is not a git-annex problem, but more likely a way some software handles symlinks and/or do file-type detection.

my setup is a following:
- multiple pc/laptop (linux)
- one NAS (debian) - main repo for music/films/photos - all in git-annex
- a Raspberry Pi (Raspbmc) - connected to the TV

I want to share all my films/music/.. to the Pi.
So, i setup a r/o NFS and mounted that in the XBMC.
So far this works, I see directories but XBMC do not recognize the media files.
It works, if I "git-annex edit ..." the file.

My _assumption_ is, that it follows the symlink, finds a file with no extension - and ignores that.
In XBMC-config there is a list of supported filetypes by extension. ( .avi,.mpg,.foo,.bar )

What I have think of so far:
- tell XBMC somehow to load all the files (did not work)
- having some kind of (FUSE?) filter, which hides the symlink in a transparent way
- creating via script hard-links in a seperate folder with same structure, mount that.
- using some alternative to NFS ( like ftp, smb ) or a other kind of media-server (server-side)

any comments, ideas ?
If i find a solution, I'll post it here.

.ka


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawmBUR4O9mofxVbpb8JV9mEbVfIYv670uJo"
 nickname="Justin"
 subject="Samba works"
 date="2013-05-30T14:22:46Z"
 content="""
I use samba. Just need to set

    unix extensions = no

which causes samba to handle the symlinks internally.
"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 nickname="joey"
 subject="comment 2"
 date="2013-05-30T15:20:51Z"
 content="""
The default git-annex backend for over a year now is SHA256E, which includes the filename extension in the key to deal with programs that rely on them. If you're not using that backend, you can `git annex migrate` to it.

Or you can turn on direct mode, which will certainly solve the problem.
"""]]


code:
[[!comment format=mdwn
 username="ka7"
 ip="2001:7b8:155d:0:222:64ff:fe16:dc52"
 subject="comment 3"
 date="2013-05-31T15:45:51Z"
 content="""
ah, great.
I'll try the samba way.
And i guess my git-annex is too old (using the git-annex version: 3.20120629~bpo60+2 from debian-backports)
thanks !
"""]]


code:
[[!comment format=mdwn
 username="ka7"
 ip="2001:7b8:155d:0:222:64ff:fe16:dc52"
 subject="comment 4"
 date="2013-05-31T18:46:29Z"
 content="""
after a quick check I realized that sha256E is available -- migrate and yepey!
"""]]


code:
Hi,

Not sure if this is a bug or not, so posting it here rather than in the bug tracker.

Basically, I am unable to add an XMPP identity via the webapp, I continually  get the error message: "Unable to connect to the Jabber server. Maybe you entered the wrong password? (Error message: AuthenticationFailure)"

I didn't enter the wrong password, I copy/pasted it from my password manager and the same account/password work fine with Pidgin. The log doesn't show anything of interest that I can see.

Using git annex version 4.20130405 from Debain Unstable. The server is ejabberd running on my VPS with a self signed certificate.

Anyone got any idea what might be going on?

Cheers,

Rob


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 nickname="joey"
 subject="comment 1"
 date="2013-04-09T18:31:01Z"
 content="""
I installed ejabberd and was able to trivially reproduce this bug. Pidgin works, git-annex fails, and I just created the user with `ejabberdctl register`. Filed an upstream bug report against the haskell network-protocol-xmpp library, by email, since there's no upstream BTS:
"""]]


code:
[[!comment format=mdwn
 username="robconnolly"
 ip="203.167.218.84"
 subject="comment 2"
 date="2013-04-09T23:01:44Z"
 content="""
Excellent, thanks for the quick response/action!
"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 nickname="joey"
 subject="comment 3"
 date="2013-04-12T22:25:12Z"
 content="""
John has gotten back to me with some analysis. The authentication failure is when SCRAM-SHA-1 is used. Ejabberd is rejecting the client's authentication request with a reason of \"bad-protocol\". It's not clear if ejabberd is broken or if gsasl is generating a bad SCRAM-SHA-1 authentication. We're inclined toward the latter, and will be forwarding this on to ejabberd.

The only way to turn off this authentication in ejabberd is to configure it to store passwords in plain text, or downgrade to a version older than 2.1.9, which first added it. Note that debian stable contains 2.1.5.3, so is not currently affected, for example.
"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 nickname="joey"
 subject="comment 4"
 date="2013-04-17T13:26:45Z"
 content="""
Upstream bug report, with patch https://support.process-one.net/browse/EJAB-1632
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawkF8_uQjLYm5Mf5F_JuVW-BxlvzpWjvR_o"
 nickname="Andrew"
 subject="Wheezy is affected"
 date="2013-05-25T12:00:21Z"
 content="""
Now that Wheezy is out, Debian Stable appears to be affected by this issue. It has ejabberd v2.1.10 .

The patch on the linked bug report doesn't apply cleanly to v2.1.10. I'll try and look at this again tomorrow.
"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 nickname="joey"
 subject="comment 6"
 date="2013-05-25T17:57:43Z"
 content="""
Debian has it fixed in 2.1.10-5 but this has unfortunately not made it to the wheezy release.
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawkF8_uQjLYm5Mf5F_JuVW-BxlvzpWjvR_o"
 nickname="Andrew"
 subject="comment 7"
 date="2013-05-25T20:27:32Z"
 content="""
Ah ha, so it is. I've installed ejabberd from Jessie onto my Wheezy box and now git-annex can authenticate.

Thank you!
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawnVTPaZaIfnZqDbkmW97pqxwi2QuZIJAF4"
 nickname="Boris"
 subject="jabberd also affected"
 date="2013-06-25T13:00:46Z"
 content="""
I tried logging in to my local 'jabberd' server and got the same response:

AuthenticationError \"No supported authentication mechanism\"

checking the version gives:

    jabberd -v
    jabberd14 version 1.6.1.1

    The following optional features have been enabled:
    - support for IPv6.
    - support for TLS using GNU TLS
    - support for MySQL
    - support for PostgreSQL

--------------
Can I do anything, investigate something?
"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="4.152.108.193"
 subject="comment 9"
 date="2013-06-25T17:18:56Z"
 content="""
Boris, that seems like an unrelated problem to the one affecting ejabberd. Different error message.

"""]]


code:
git-annex-assistant is so exciting...

Can't set up XMPP sharing. The web app says "Wrong Password" for my gmail/home-google-apps/work-google-apps addresses, but they are correct.

I'd love to send some debug output, can anyone tell me how to get it? Of course if I'm just doing something wrong and you have an FAQ link or similar, feel free to RTFM me :)

I'm on Ubuntu 12.04 running https://downloads.kitenet.net/git-annex/linux/3.20121112/git-annex-standalone-amd64.tar.gz

Carlo


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawn7gQ1zZDdWhXy9H51W2krZYShNmKL3qfM"
 nickname="Karsten"
 subject="comment 1"
 date="2012-11-13T14:38:28Z"
 content="""
Just an idea: Do you have two-factor-authorisation set up for your google account, you need to generate an application specific password, as the 'normal' password will not work.
"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="4.252.11.120"
 subject="comment 2"
 date="2012-11-13T17:21:35Z"
 content="""
@Karsten that seems likely.

I've made a change so it shows the error message from the XMPP library I'm using. Which may or may not be useful..
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawnZEanlyzay_QlEAL0CWpyZcRTyN7vay8U"
 nickname="Carlo"
 subject="XMPP"
 date="2012-11-13T23:26:50Z"
 content="""
Just tried a fresh jabber.org address, same error.

Will see if I can get the changed version built. Or are there nightlies?
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawnZEanlyzay_QlEAL0CWpyZcRTyN7vay8U"
 nickname="Carlo"
 subject="Now Works! For jabber.org, gmail, but NOT google apps"
 date="2012-11-27T11:40:43Z"
 content="""
Hi again!

With the latest version, 3.20121126, pairing works for both something@gmail.com and something@jabber.org! Yay!

What still fails is my google apps address, something@somedomain.com, seems to behave same as before. the assistant says

  Unable to connect to the Jabber server. Maybe you entered the wrong password? (Error message: connect: timeout (Connection timed out))

If you would like a google apps address on my domain, so you don't have to set up your own, just let me know.

"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="4.154.6.49"
 subject="comment 5"
 date="2012-11-27T20:32:47Z"
 content="""
Hmm, I can't think of anything I've changed that would make it work where it didn't before.

I suspect the google apps problem is because it looks for a SRV record in DNS to find the jabber server, and something is wrong there. Without knowing what \"something@somedomain.com\" is standing for, it's hard to tell. :)

If you'd like to set me up something to test with, my email address is id@joeyh.name
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawnZEanlyzay_QlEAL0CWpyZcRTyN7vay8U"
 nickname="Carlo"
 subject="comment 6"
 date="2012-11-29T15:37:46Z"
 content="""
Weird, 20121112 works too now.

Looks like some kind of edge case... I'll let you know if I find it.

"""]]