code:
### Please describe the problem.
Git-annex will constantly sync most(if not all) my files to box.com

### What steps will reproduce the problem?
1 - Use git-annex instead of Dropbox at work
2 - Boot computer.
3 - Watch it sync everything to box.com (even files i believe it has transferred each and every day for the last few months)

### What version of git-annex are you using? On what operating system?
git-annex version: 4.20130827

But i have never seen it work satisfactory in any version.

Also, i have seen this is 10+ different clean git-annexes. So it isn't annex specific.

### Please provide any additional information below.

I am going to add more debug to this bug constantly. (I intend to do a full 'git annex copy --to box.com --not --in box.com' daily, and see if the same files are transfered again and again)

For now, i see a few different issues already:

[[!format sh """
# If you can, paste a complete transcript of the problem occurring here.
# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log

tou@DSK1049:~/work-annex$ git annex copy --to box.com --not --in box.com 2>&1 | tee ../work-annex-copy-to-box.com-not-in-box.com-run1.log
[2013-09-11 09:24:53 CEST] read: git ["--git-dir=/home/tou/work-annex/.git","--work-tree=/home/tou/work-annex","show-ref","git-annex"]
[2013-09-11 09:24:53 CEST] read: git ["--git-dir=/home/tou/work-annex/.git","--work-tree=/home/tou/work-annex","show-ref","--hash","refs/heads/git-annex"]
[2013-09-11 09:24:53 CEST] read: git ["--git-dir=/home/tou/work-annex/.git","--work-tree=/home/tou/work-annex","log","refs/heads/git-annex..dbe8b1cfa5f84126c45a39fdc7c7f26e272c71cc","--oneline","-n1"]
[2013-09-11 09:24:53 CEST] read: git ["--git-dir=/home/tou/work-annex/.git","--work-tree=/home/tou/work-annex","log","refs/heads/git-annex..45e279375897a2cd7f5b893402e0ec25c1b23436","--oneline","-n1"]
[2013-09-11 09:24:54 CEST] read: git ["--git-dir=/home/tou/work-annex/.git","--work-tree=/home/tou/work-annex","log","refs/heads/git-annex..c4921be4434f751493fce1c932ac759214abacd4","--oneline","-n1"]
[2013-09-11 09:24:54 CEST] read: git ["--git-dir=/home/tou/work-annex/.git","--work-tree=/home/tou/work-annex","log","refs/heads/git-annex..d591398dc1cac824a5fc5bdacdcb82301a9b15a3","--oneline","-n1"]
[2013-09-11 09:24:54 CEST] chat: git ["--git-dir=/home/tou/work-annex/.git","--work-tree=/home/tou/work-annex","cat-file","--batch"]
[2013-09-11 09:24:54 CEST] read: git ["config","--null","--list"]
[2013-09-11 09:24:54 CEST] read: git ["--git-dir=/home/tou/work-annex/.git","--work-tree=/home/tou/work-annex","ls-files","--cached","-z","--"]
[2013-09-11 09:24:54 CEST] chat: git ["--git-dir=/home/tou/work-annex/.git","--work-tree=/home/tou/work-annex","cat-file","--batch"]
copy Documents/Gamle catillo overførsler/Rapport b-bm.odt (gpg) (checking box.com...) ok
copy Documents/Gamle catillo overførsler/Rapport brandt skorstensfejeren.odt (checking box.com...) (failed to read https://www.box.com/dav/work-annex/4a5/18e/GPGHMACSHA1--5f8660edac93899cf9adc5fadcc480ddc2992bb1/GPGHMACSHA1--5f8660edac93899cf9adc5fadcc480ddc2992bb1.chunkcount) failed
copy Documents/Gamle catillo overførsler/Rapport teamkoege.odt (checking box.com...) (failed to read https://www.box.com/dav/work-annex/98d/ae7/GPGHMACSHA1--5253241407527aa6c980f1174fdbc32713c54c44/GPGHMACSHA1--5253241407527aa6c980f1174fdbc32713c54c44.chunkcount) failed
copy Documents/Gamle catillo overførsler/Rapport vikarborsen.odt (checking box.com...) ok
copy Documents/Gamle catillo overførsler/Rapport-terrariemesteren.odt (checking box.com...) ok
copy Documents/Gamle catillo overførsler/catillo-efhandel.odt (checking box.com...) (failed to read https://www.box.com/dav/work-annex/9d9/aea/GPGHMACSHA1--1516eac1ec7b4ceaa840faebabde1f50f5db0a52/GPGHMACSHA1--1516eac1ec7b4ceaa840faebabde1f50f5db0a52.chunkcount) failed
copy Documents/Nøgeordsanalyse catillo104.xlsx (checking box.com...) (ResponseTimeout) failed
copy Documents/Søgeordsliste.txt (checking box.com...) ok
copy Documents/catillo guide.odt (checking box.com...) ok
copy Documents/guide bruger oprettelse.odt (checking box.com...) (failed to read https://www.box.com/dav/work-annex/49e/175/GPGHMACSHA1--2b47737f8de7faac7704eaa322785edad63a921c/GPGHMACSHA1--2b47737f8de7faac7704eaa322785edad63a921c.chunkcount) failed
copy Documents/guide skift backup bånd.odt (checking box.com...) ok
copy Documents/lilletest.csv (checking box.com...) (failed to read https://www.box.com/dav/work-annex/915/373/GPGHMACSHA1--49ba3d5f63c012ae2cd2c0fc3e729178b1023c33/GPGHMACSHA1--49ba3d5f63c012ae2cd2c0fc3e729178b1023c33.chunkcount) failed
copy Dropbox/adams-scraper/CommonFunctions.py (checking box.com...) (ResponseTimeout) failed
copy Dropbox/adams-scraper/__pycache__/CommonFunctions.cpython-33.pyc (checking box.com...) (to box.com...) [2013-09-11 09:28:30 CEST] chat: gpg ["--batch","--no-tty","--use-agent","--quiet","--trust-model","always","--batch","--passphrase-fd","14","--symmetric","--force-mdc"]

100%          0.0 B/s 0sResponseTimeout
ResponseTimeout
failed

# End of transcript or log.
"""]]

More to come(full log)

> This is [[fixed|done]] in git; when built with a new enough
> version of the haskell DAV library, git-annex disables the default 5
> second timeout.
>
> It'll still be present in the Debian stable backports, which are
> built with an old version of DAV. Not much I can do about that;
> backporting DAV would be difficult.
>
> The daily builds are updated to use the new version.
> --[[Joey]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="4.154.4.51"
 subject="comment 1"
 date="2013-09-12T20:29:14Z"
 content="""
It seems you are getting a lot of timeouts from box.com, both when checking if content is present there and when uploading content that it does not have.

I have heard some grumbles about box.com not being very reliable right now.
"""]]

repo_name: kakafarm/git

license: unknown


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawmkBwMWvNKZZCge_YqobCSILPMeK6xbFw8"
 nickname="develop"
 subject="comment 2"
 date="2013-09-12T20:54:33Z"
 content="""
The problem being. I've used box.com with one of my own hooks(owncloud), where it is completely stable!

The instability is not a the box.com end.

"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="4.154.4.51"
 subject="comment 3"
 date="2013-09-12T20:56:44Z"
 content="""
git-annex is using box.com's WebDAV interface. Is owncloud?
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawmkBwMWvNKZZCge_YqobCSILPMeK6xbFw8"
 nickname="develop"
 subject="comment 4"
 date="2013-09-12T21:33:29Z"
 content="""
It uses the following url https://www.box.com/dav

"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="4.154.4.51"
 subject="comment 5"
 date="2013-09-12T21:44:32Z"
 content="""
It may be that there is a timeout in the http library that I am using for webdav that is too low. See <https://github.com/snoyberg/http-conduit/issues/137>
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawmkBwMWvNKZZCge_YqobCSILPMeK6xbFw8"
 nickname="develop"
 subject="comment 6"
 date="2013-09-12T21:49:05Z"
 content="""
That would probably do it. With other implementations(fuse mount) i've seen stalls and other bad behaviour with that DAV server.

The owncloud hook doesn't have a timeout set. It probably should have one to prevent a total stall.
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawmkBwMWvNKZZCge_YqobCSILPMeK6xbFw8"
 nickname="develop"
 subject="Logs"
 date="2013-09-13T11:43:22Z"
 content="""
So have two days of logs now, and it doesn't look like it is retrying old files.

http://paste.ubuntu.com/6101255/ <- day 1

http://paste.ubuntu.com/6101256/ <- day 2

Setting the computer to do the following loop over the weekend, lets see if everything is done come monday.

for i in `seq 3 2000`; do git annex copy --to box.com --not --in box.com 2>&1 | tee ../work-annex-copy-to-box.com-not-in-box.com-run$i.log; done

"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawmWg4VvDTer9f49Y3z-R0AH16P4d1ygotA"
 nickname="Tobias"
 subject="Update"
 date="2013-09-18T11:17:33Z"
 content="""
Only just got back to work now. The script has done 10 iteration. This is the status:

tou@DSK1049:~$ cat work-annex-copy-to-box.com-not-in-box.com-run1.log |grep \" ok\"| wc -l
137

tou@DSK1049:~$ cat work-annex-copy-to-box.com-not-in-box.com-run2.log |grep \" ok\"| wc -l
77

tou@DSK1049:~$ cat work-annex-copy-to-box.com-not-in-box.com-run3.log |grep \" ok\"| wc -l
116

tou@DSK1049:~$ cat work-annex-copy-to-box.com-not-in-box.com-run4.log |grep \" ok\"| wc -l
70

tou@DSK1049:~$ cat work-annex-copy-to-box.com-not-in-box.com-run5.log |grep \" ok\"| wc -l
26

tou@DSK1049:~$ cat work-annex-copy-to-box.com-not-in-box.com-run6.log |grep \" ok\"| wc -l
6

tou@DSK1049:~$ cat work-annex-copy-to-box.com-not-in-box.com-run7.log |grep \" ok\"| wc -l
6

tou@DSK1049:~$ cat work-annex-copy-to-box.com-not-in-box.com-run8.log |grep \" ok\"| wc -l
0

tou@DSK1049:~$ cat work-annex-copy-to-box.com-not-in-box.com-run9.log |grep \" ok\"| wc -l
0

tou@DSK1049:~$ cat work-annex-copy-to-box.com-not-in-box.com-run10.log |grep \" ok\"| wc -l
0

tou@DSK1049:~$ cat work-annex-copy-to-box.com-not-in-box.com-run11.log |grep \" ok\"| wc -l
0

tou@DSK1049:~$ cat work-annex-copy-to-box.com-not-in-box.com-run12.log |grep \" ok\"| wc -l
0

I have all the log files of course.

tou@DSK1049:~/work-annex$ git annex find --not --in box.com| wc -l
1639

So, of the last 5 iteration 0 files of 1639 missing were transfered. Most of these files are <10kbyte

http://paste.ubuntu.com/6123459/ <- pastebin of work-annex-copy-to-box.com-not-in-box.com-run11.log if it is of any use.

"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="4.153.8.80"
 subject="comment 9"
 date="2013-09-28T19:34:42Z"
 content="""
Filed a bug on the DAV library about this: <http://bugs.debian.org/724856>
"""]]


code:
### Please describe the problem.

`git annex map` shows broken graph when direct mode is used

### What steps will reproduce the problem?

See transcript

### What version of git-annex are you using? On what operating system?

[[!format sh """
$ git annex version
git-annex version: 5.20140107
build flags: Assistant Webapp Pairing Testsuite S3 WebDAV Inotify DBus XMPP DNS Feeds Quvi TDFA CryptoHash
key/value backends: SHA256E SHA1E SHA512E SHA224E SHA384E SKEIN256E SKEIN512E SHA256 SHA1 SHA512 SHA224 SHA384 SKEIN256 SKEIN512 WORM URL
remote types: git gcrypt S3 bup directory rsync web webdav glacier hook external

$ LANG=C apt-cache policy git-annex
git-annex:
  Installed: 5.20140107
  Candidate: 5.20140107
  Version table:
 *** 5.20140107 0
         50 http://http.debian.net/debian/ sid/main amd64 Packages
        100 /var/lib/dpkg/status
"""]]

### Please provide any additional information below.

[[!format sh """
# If you can, paste a complete transcript of the problem occurring here.
# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log

git init a
cd a
git annex init

cd ..
git clone a b
cd b
git annex init

# OK
git annex map

cd ../a
git annex direct
cd ../b

# NOT OK
git annex map

# End of transcript or log.
"""]]

> [[fixed|done]]; thanks for a good test case. --[[Joey]]


code:
### Please describe the problem.

Building from git fails in Assistant/WebApp/Gpg.hs

### What steps will reproduce the problem?

Follow instructions at <http://git-annex.branchable.com/install/cabal/> for building from git:

[[!format sh """
cabal update
PATH=$HOME/bin:$PATH
cabal install c2hs --bindir=$HOME/bin
cabal install --only-dependencies
cabal configure
cabal build
"""]]

### What version of git-annex are you using? On what operating system?

git master (7da58f3ebf8), openSUSE 12.3

### Please provide any additional information below.

[[!format txt """
# If you can, paste a complete transcript of the problem occurring here.
# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log

Assistant/WebApp/Configurators/Fsck.hs:69:35:
    Warning: This binding for `u' shadows the existing binding
               bound at Assistant/WebApp/Configurators/Fsck.hs:60:17

Assistant/WebApp/Configurators/Fsck.hs:126:11:
    Warning: This binding for `check' shadows the existing binding
               imported from `Assistant.WebApp.Common' at Assistant/WebApp/Configurators/Fsck.hs:16:1
-30
               (and originally defined in `Yesod.Form.Functions')
[322 of 371] Compiling Assistant.TransferQueue ( Assistant/TransferQueue.hs, dist/build/git-annex/git
-annex-tmp/Assistant/TransferQueue.o )
[323 of 371] Compiling Assistant.Threads.Glacier ( Assistant/Threads/Glacier.hs, dist/build/git-annex
/git-annex-tmp/Assistant/Threads/Glacier.o )
[324 of 371] Compiling Assistant.Threads.Merger ( Assistant/Threads/Merger.hs, dist/build/git-annex/g
it-annex-tmp/Assistant/Threads/Merger.o )
[325 of 371] Compiling Assistant.Threads.Cronner ( Assistant/Threads/Cronner.hs, dist/build/git-annex
/git-annex-tmp/Assistant/Threads/Cronner.o )
[326 of 371] Compiling Assistant.MakeRemote ( Assistant/MakeRemote.hs, dist/build/git-annex/git-annex
-tmp/Assistant/MakeRemote.o )
[327 of 371] Compiling Assistant.WebApp.Gpg ( Assistant/WebApp/Gpg.hs, dist/build/git-annex/git-annex
-tmp/Assistant/WebApp/Gpg.o )

Assistant/WebApp/Gpg.hs:88:40:
    Not in scope: type constructor or class `MonadIO'
    Perhaps you meant `Monad' (imported from Assistant.WebApp.Common)

# End of transcript or log.
"""]]

> [[fixed|done]] --[[Joey]]


code:
Here goes...

<pre>
laplace:git-annex jtang$ make
ghc -O2 -threaded -Wall -ignore-package monads-fd -ignore-package monads-tf -outputdir tmp -IUtility -DWITH_ASSISTANT -DWITH_S3 -DWITH_WEBAPP -DWITH_KQUEUE -DOSX --make git-annex Utility/libdiskfree.o Utility/libmounts.o Utility/libkqueue.o

Assistant/Threads/MountWatcher.hs:40:0:
     warning: #warning Building without dbus support; will use mtab polling
[165 of 208] Compiling Assistant.Alert  ( Assistant/Alert.hs, tmp/Assistant/Alert.o )
[173 of 208] Compiling Assistant.DaemonStatus ( Assistant/DaemonStatus.hs, tmp/Assistant/DaemonStatus.o )
[174 of 208] Compiling Assistant.TransferQueue ( Assistant/TransferQueue.hs, tmp/Assistant/TransferQueue.o )
[175 of 208] Compiling Assistant.Threads.Watcher ( Assistant/Threads/Watcher.hs, tmp/Assistant/Threads/Watcher.o )

Assistant/Threads/Watcher.hs:61:43:
    Couldn't match expected type `Utility.Kqueue.Kqueue'
                with actual type `()'
    Expected type: IO Utility.Kqueue.Kqueue -> IO Utility.Kqueue.Kqueue
      Actual type: IO Utility.Kqueue.Kqueue -> IO ()
    In the fourth argument of `watchDir', namely `startup'
    In the second argument of `($)', namely
      `watchDir "." ignored hooks startup'
make: *** [git-annex] Error 1
</pre>

> [[fixed|done]] --[[Joey]]


code:
Building commit 8baff14054e65ecbe801eb66786a55fa5245cb30 yields this...

[164 of 189] Compiling Command.Sync ( Command/Sync.hs, tmp/Command/Sync.o )
Command/Sync.hs:268:34:
Not in scope: `vermarker'
Perhaps you meant `varmarker' (line 267)
make: *** [git-annex] Error 1

Supplied fix...

From a23a1af99c7a95c316a87f9c6f5f67a6f8ff6937 Mon Sep 17 00:00:00 2001
From: Jimmy Tang <jtang@tchpc.tcd.ie>
Date: Wed, 27 Jun 2012 21:55:22 +0100
Subject: [PATCH 14/14] fix build issue introduced in
 8baff14054e65ecbe801eb66786a55fa5245cb30

---
 Command/Sync.hs | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Command/Sync.hs b/Command/Sync.hs
index b2bf24d..dfaed59 100644
--- a/Command/Sync.hs
+++ b/Command/Sync.hs
@@ -265,7 +265,7 @@ mergeFile file key
        | otherwise = go $ shortHash $ show key
        where
                varmarker = ".variant-"
-               doubleconflict = vermarker `isSuffixOf` (dropExtension file)
+               doubleconflict = varmarker `isSuffixOf` (dropExtension file)
                go v = takeDirectory file
                        </> dropExtension (takeFileName file)
                        ++ varmarker ++ v
--
1.7.11.1

[[fixed|done]]


code:
### Please describe the problem.
In "Configuring jabber account" custom domains through google apps don't connect properly.

### What steps will reproduce the problem?
Try to use a google account that uses a non-gmail domain, e.g. user@domain.com

### What version of git-annex are you using? On what operating system?
Newest, on Mac OS 10.9

### Please provide any additional information below.
The issue is because git-annex is trying to connect to @domain.com as the jabber server, but the server should be talk.google.com:5223.

[[!format sh """
# If you can, paste a complete transcript of the problem occurring here.
# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log

# End of transcript or log.
"""]]

[[!tag moreinfo]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="108.236.230.124"
 subject="comment 1"
 date="2014-07-02T17:28:01Z"
 content="""
This seems like the same problem as in [[bugs/googlemail]].

git-annex uses SRV records to find the jabber server for a domain. What is google doing that makes this not work? Do you have a sample domain?
"""]]


code:
### Please describe the problem.

### What steps will reproduce the problem?

[[!format sh """
$> git annex get 2read/ISNN2010__Tang.pdf
git-annex: Cannot mix --all or --unused with file names.
"""]]

### What version of git-annex are you using? On what operating system?

[[!format sh """
$> apt-cache policy git-annex
git-annex:
  Installed: 5.20140116
  Candidate: 5.20140116
  Version table:
 *** 5.20140116 0
        600 http://debian.lcs.mit.edu/debian/ sid/main amd64 Packages
        100 /var/lib/dpkg/status
"""]]

### Please provide any additional information below.

[[!format sh """
# If you can, paste a complete transcript of the problem occurring here.
# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log

$> git annex get 2read/ISNN2010__Tang.pdf
git-annex: Cannot mix --all or --unused with file names.

but seems to start fetching some load if I do not specify any path and just run 'git annex get'.

There seems to be some screw up:

I have plenty of objects under .git/annex/objects/ (seems largely from
the directory above), nothing is now reported by unused (with obscure
msg):

$> du -scmL * 2>/dev/null | tail -1
1	total

$> du -scm .git/annex/objects
334	.git/annex/objects
334	total

$> git annex dropunused all
git-annex: Map.findMin: empty map has no minimal element

Here is some portion of the history which lead to such a state (there
was git annex unused somewhere before)

25954  git annex move --unused --to onerussian.com_annex
25955  git annex dropunused
25956  git annex dropunused all
25962  git annex unused
25963  git log --stat -SSHA256E-s5639442--67691e57cb4d6c51afe838590ad265ba4bea9c291cf52d58ed24f05b70bf33bf.mp3
25965  git log --stat -SSHA256E-s143042--b4012bf03ed0a387a9e714390efa75f1dd769162cca4c9b77e516732342be3f9.html
25968  git annex move --unused --to onerussian.com_annex
25969  git annex dropunused all
25976  git annex unused
25978  git br
25980  git log --stat -Ss741707--7c215090893f1f0c994e2a9ad3088016676464bbad26768841dd08c07295a2fe.pdf.map
25981  git annex unused
25982  git annex fsck
25983  git annex unused
25984  git annex dropkey
25985  git log --stat -SSHA256E-s14534131--20de680eedb3e1fb687c9b00c154d978333b61f4ea122c632bdb5bcdbb1553ff.pdf
25986  git show de3ccae8304efbae4a7a8add49de638f64b821fc
25991  git annex fsck

# End of transcript or log.
"""]]

[[!tag moreinfo]]

> Tagged moreinfo since I have a workable theory about how this happened,
> which would make it user configuration error and not a bug, but
> that has not been confirmed. --[[Joey]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawnx8kHW66N3BqmkVpgtXDlYMvr8TJ5VvfY"
 nickname="Yaroslav"
 subject="git annex fix starts fixing but then spits bulk of errors"
 date="2014-01-18T05:42:15Z"
 content="""
probably related:

```
fix books/Мои первые книжки/PDF/Благинина Е.А. - Лодочки (Мои первые книжки) - 1962.pdf ok
fix books/Мои первые книжки/PDF/Благинина Е.А. - Не мешайте мне трудиться (Мои первые книжки) - 1975.pdf fatal: This operation must be run in a work tree
ok
(Recording state in git...)

git-annex: user error (xargs [\"-0\",\"git\",\"--git-dir=/home/yoh/annex/.git\",\"add\",\"--force\",\"--\"] exited 123)
fatal: This operation must be run in a work tree
failed
(Recording state in git...)

git-annex: user error (xargs [\"-0\",\"git\",\"--git-dir=/home/yoh/annex/.git\",\"add\",\"--force\",\"--\"] exited 123)
fatal: This operation must be run in a work tree
failed
(Recording state in git...)

....
```
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawnx8kHW66N3BqmkVpgtXDlYMvr8TJ5VvfY"
 nickname="Yaroslav"
 subject="repair seems to be also confused"
 date="2014-01-18T05:47:02Z"
 content="""
[[[
$> git annex repair
Running git fsck ...
No problems found.
fatal: '/home/yoh/annex/.git' is outside repository
Had to delete the .git/annex/index file as it was corrupt.
No data was lost.
ok

$> ls
2enjoy/   2read/    2watch/  books/  hardware/  videos/
2listen/  2review/  abooks/  docs/   pics/

$> git annex repair
Running git fsck ...
No problems found.
fatal: '/home/yoh/annex/.git' is outside repository
Had to delete the .git/annex/index file as it was corrupt.
No data was lost.
ok

$> git annex get 2read/ISNN2010__Tang.pdf
git-annex: Cannot mix --all or --unused with file names.

]]]
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawnx8kHW66N3BqmkVpgtXDlYMvr8TJ5VvfY"
 nickname="Yaroslav"
 subject="could it be part/reason of the problem"
 date="2014-01-18T06:05:50Z"
 content="""
not sure how that happened... definitely not me consciously! ;-)  some commands are complaining that \"You cannot run this command in a bare repository\" which I thought is BS since it is not BARE! but then looked into .git/config and it does have core.bare = True ... yikes!..

This repository is also under assistant \"control\".

changing to bare=False seems to start 'get'ing things, git annex repair doesn't produce obscure errors.

git annex fix though now doesn't report any problems -- only 'ok', but none of those files mentioned 'ok' has a working symlink,,,  but I guess that is a fluke after many upgrades -- just dropping everything locally and getting needed context after purging .git/annex/objects .

So I guess issue is resolved by discovering that repository was set to 'bare' mode somehow although it was not and seemed like working but not quite
"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="209.250.56.43"
 subject="comment 4"
 date="2014-01-18T15:42:59Z"
 content="""
git-annex sets core.bare=true for direct mode, but it also then sets annex.direct=true and so does not treat it as a bare mode repository. If you had eg, manually tried to change annex.direct to false, and left it in bare mode, that would explain everything.

> git annex fix though now doesn't report any problems -- only 'ok', but none of those files mentioned 'ok' has a working symlink

That is completely normal behavior; git annex fix does not care if the content is locally present or not; it just checks that the symlinks would point to it if it were present.

(Fixed the partial function in dropunused.)
"""]]


code:
### Please describe the problem.
Then starting git-annex on my Galaxy Nexus Android device, in the terminal window I get:

[[!format sh """
Falling back to hardcoded app location: cannot find expected files in /data/app-lib
git annex webapp
u0_a123@maguro:/sdcad/git-annex.home $ git annex webapp
CANNOT LINK EXECUTABLE: git-annex invalid R_ARM_COPY relocation against DT_SYMBOLIC shared library libc.so (built with -Bsymbolic?)
u0_a123@maguro:/sdcad/git-annex.home $
"""]]

### What steps will reproduce the problem?
Start git-annex.

### What version of git-annex are you using? On what operating system?
Nightly build and release from a few days ago - 1.0.52

### Please provide any additional information below.

[[!format sh """
# If you can, paste a complete transcript of the problem occurring here.
# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log

# End of transcript or log.
"""]]

> [[dup|done]] of [[git-annex_broken_on_Android_4.3]].--[[Joey]]


code:
What steps will reproduce the problem?

> Building git-annex on the ghc7.0 branch on a Mac with the default case-insensitive file system

What is the expected output? What do you see instead?

> Expected: build successfully; instead:

    ld: duplicate symbol _UtilityziDiskFree_zdwa_info in dist/build/git-annex/git-annex-tmp/Utility/diskfree.o and dist/build/git-annex/git-annex-tmp/Utility/DiskFree.o for architecture x86_64

What version of git-annex are you using? On what operating system?

> commit `0bd5c90ef0518f75d52f0c5889422d8233df847d` on a Mac OS 10.6 and 10.7, using the Haskell Platform 2012.04

Please provide any additional information below.

> The problem is that since `DiskFree.hs` generates `DiskFree.o` and `diskfree.c` generates `diskfree.o`, a case-insensitive file system overwrites one object file with the other.  Renaming `diskfree.c` to `diskfreec.c` and changing the corresponding filenames in `git-annex.cabal` fixes the problem.

>> Man, not this again. The 80's called, they want their
>> unix portability wars back. [[fixed|done]]. --[[Joey]]


code:
What steps will reproduce the problem?

    $ git init a.git
    Initialized empty Git repository in /var/tmp/git-annex-bug/a.git/.git/
    $ cd a.git
    $ git annex init a
    init a ok
    (Recording state in git...)
    $ touch Ren$'\351'
    $ git annex add Ren$'\351'
    add René (checksum...) ok
    (Recording state in git...)
    $ git ci -m "Added Rene'."
    [master (root-commit) a61b796] Added Rene'.
     1 files changed, 1 insertions(+), 0 deletions(-)
     create mode 120000 "Ren\351"
    $ cd ..
    $ git clone -o a a.git b.git
    Cloning into b.git...
    remote: Counting objects: 13, done.
    remote: Compressing objects: 100% (9/9), done.
    remote: Total 13 (delta 1), reused 0 (delta 0)
    Receiving objects: 100% (13/13), done.
    Resolving deltas: 100% (1/1), done.
    $ cd b.git
    $ git annex copy --from=a --fast -v
    (merging a/git-annex into git-annex...)
    copy René
    git-annex: /var/tmp/git-annex-bug/b.git/.git/annex/transfer/download/7c5ee764-e8c6-11e1-b0c5-67c6ec1236d6/SHA256-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855: commitBuffer: invalid argument (invalid character)
    failed
    (Recording state in git...)
    git-annex: copy: 1 failed

What is the expected output? What do you see instead?

Expect that files will be copied, but they are not.

What version of git-annex are you using? On what operating system?

    $ echo $LANG
    en_US.UTF-8
    $ lsb_release -a
    No LSB modules are available.
    Distributor ID:	Ubuntu
    Description:	Ubuntu 11.10
    Release:	11.10
    Codename:	oneiric
    $ uname -a
    Linux pdx-desktop 3.0.0-23-generic #39-Ubuntu SMP Thu Jul 19 19:18:53 UTC 2012 i686 i686 i386 GNU/Linux
    $ bash --version
    GNU bash, version 4.2.10(1)-release (i686-pc-linux-gnu)
    Copyright (C) 2011 Free Software Foundation, Inc.
    License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>

    This is free software; you are free to change and redistribute it.
    There is NO WARRANTY, to the extent permitted by law.
    $ ghc --version
    The Glorious Glasgow Haskell Compilation System, version 7.4.2
    $ git annex version
    git-annex version: 3.20120807
    local repository version: 3
    default repository version: 3
    supported repository versions: 3
    upgrade supported from repository versions: 0 1 2

Please provide any additional information below.

The problem is related to weird characters in file names.  In the
above example, the "weird character" is an accented 'e' (entered with
$'\351' in bash and zsh).  I am able to add the files with weird
characters in their name to one annex, but I cannot copy them to other
annexes using `git annex copy`.

The above example is a simplification of a real problem I am
experiencing.  In my real scenario, the file is not empty.  See the
attachment for some variations, including with non-empty
files. UPDATE: I'm not allowed to add attachments. See below.

May be related to these (long-ago fixed) bugs:
http://git-annex.branchable.com/todo/support-non-utf8-locales/

"Attachment": Here are my notes, including more examples:

    Programs I'm using:

    $ echo $LANG
    en_US.UTF-8
    $ lsb_release -a
    No LSB modules are available.
    Distributor ID:	Ubuntu
    Description:	Ubuntu 11.10
    Release:	11.10
    Codename:	oneiric
    $ uname -a
    Linux pdx-desktop 3.0.0-23-generic #39-Ubuntu SMP Thu Jul 19 19:18:53 UTC 2012 i686 i686 i386 GNU/Linux
    $ bash --version
    GNU bash, version 4.2.10(1)-release (i686-pc-linux-gnu)
    Copyright (C) 2011 Free Software Foundation, Inc.
    License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>

    This is free software; you are free to change and redistribute it.
    There is NO WARRANTY, to the extent permitted by law.
    $ ghc --version
    The Glorious Glasgow Haskell Compilation System, version 7.4.2
    $ git annex version
    git-annex version: 3.20120807
    local repository version: 3
    default repository version: 3
    supported repository versions: 3
    upgrade supported from repository versions: 0 1 2

    Simplest way to see problem: one empty file with weird character
    (accented e: $'\351') in name:

    $ git init a.git
    Initialized empty Git repository in /var/tmp/git-annex-bug/a.git/.git/
    $ cd a.git
    $ git annex init a
    init a ok
    (Recording state in git...)
    $ touch Ren$'\351'
    $ git annex add Ren$'\351'
    add René (checksum...) ok
    (Recording state in git...)
    $ git ci -m "Added Rene'."
    [master (root-commit) a61b796] Added Rene'.
     1 files changed, 1 insertions(+), 0 deletions(-)
     create mode 120000 "Ren\351"
    $ cd ..
    $ git clone -o a a.git b.git
    Cloning into b.git...
    remote: Counting objects: 13, done.
    remote: Compressing objects: 100% (9/9), done.
    remote: Total 13 (delta 1), reused 0 (delta 0)
    Receiving objects: 100% (13/13), done.
    Resolving deltas: 100% (1/1), done.
    $ cd b.git
    $ git annex copy --from=a --fast -v
    (merging a/git-annex into git-annex...)
    copy René
    git-annex: /var/tmp/git-annex-bug/b.git/.git/annex/transfer/download/7c5ee764-e8c6-11e1-b0c5-67c6ec1236d6/SHA256-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855: commitBuffer: invalid argument (invalid character)
    failed
    (Recording state in git...)
    git-annex: copy: 1 failed

    Problem disappears with two empty files:

    $ cd ..
    $ git init a2.git
    Initialized empty Git repository in /var/tmp/git-annex-bug/a2.git/.git/
    $ cd a2.git
    $ git annex init a2
    init a2 ok
    (Recording state in git...)
    $ touch Ren$'\351'
    $ git annex add Ren$'\351'
    add René (checksum...) ok
    (Recording state in git...)
    $ git ci -m "Add Ren$'\351'."
    [master (root-commit) 62ac1c8] Add Ren$'\351'.
     1 files changed, 1 insertions(+), 0 deletions(-)
     create mode 120000 "Ren\351"
    $ touch Rene
    $ git annex add Rene
    add Rene (checksum...) ok
    (Recording state in git...)
    $ git ci -m "Add Rene."
    [master c455523] Add Rene.
     1 files changed, 1 insertions(+), 0 deletions(-)
     create mode 120000 Rene
    $ cd ..
    $ git clone -o a2 a2.git b2.git
    Cloning into b2.git...
    done.
    $ cd b2.git
    $ git annex copy --from=a2 --fast -v
    (merging a2/git-annex into git-annex...)
    copy Rene (from a2...) ok
    (Recording state in git...)

    Problem returns with two non-empty files:

    $ cd ..
    $ git init a4.git
    Initialized empty Git repository in /var/tmp/git-annex-bug/a4.git/.git/
    $ cd a4.git
    $ git annex init a4
    init a4 ok
    (Recording state in git...)
    $ touch Ren$'\351'
    $ rm Ren$'\351'
    $ ls
    $ echo "some content" > Ren$'\351'
    $ git annex add Ren$'\351'
    add René (checksum...) ok
    (Recording state in git...)
    $ git ci -m "Add Ren$'\351'."
    [master (root-commit) f090d90] Add Ren$'\351'.
     1 files changed, 1 insertions(+), 0 deletions(-)
     create mode 120000 "Ren\351"
    $ echo "some other content" > Rene
    $ git annex add Rene
    add Rene (checksum...) ok
    (Recording state in git...)
    $ git ci -m "Add Rene."
    [master 97c20cd] Add Rene.
     1 files changed, 1 insertions(+), 0 deletions(-)
     create mode 120000 Rene
    $ cd ..
    $ git clone -o a4 a4.git b4.git
    Cloning into b4.git...
    done.
    $ cd b4.git
    $ git annex copy --from=a4 --fast -v
    (merging a4/git-annex into git-annex...)
    copy Rene (from a4...) ok
    copy René
    git-annex: /var/tmp/git-annex-bug/b4.git/.git/annex/transfer/download/a5fcd0d4-e8c8-11e1-bb41-43ce1cb9a9f1/SHA256-s13--1c87b6727f523662df714f06a94ea27fa4d9050c38f4f7712bd4663ffbfdfa01: commitBuffer: invalid argument (invalid character)
    failed
    (Recording state in git...)
    git-annex: copy: 1 failed

> [[Fixed|done]]. Sorry this took so long, I was at a very busy point when
> you filed this and am only just getting caught up. --[[Joey]]


code:
When two git-annex processes are running and both modifying the git-annex
branch, it's possible one will fail due to git's locking. When this
happens, git-annex has already recorded its state in the journal (so no
data is lost), but git-annex does crash, which can be surprising.

I feel that, in general, multiple git-annex processes should be able to run
concurrently. A big lock around all commands, or even all
repository-modifying commands is a bad idea. Also, it's probably best to
only worry about locking conflicts editing the git-annex branch. While `git
annex add` and a few other commands make changes to the main git repo,
and can have similar locking issues, so can any git commands that stage
changes (I think.. check).

Probably should KISS. Just add a lock file that is taken before changes to
the git-annex branch, and if it's locked, wait. Changes to the git-annex
branch tend to happen quickly (unless it's committing an enormous set of
changes, and even that is relatively fast), so waiting seems ok. --[[Joey]]

----

Commit 7981eb4cb512fbe3c49a3dd165c31be14ae4bc49 is more pessimistic,
describes some other potential issues.

* The journal needs to be emptied (done) and kept locked (not done) during
  a merge, since a merge operates at a level below the journal, and any
  changes that are journaled during a merge can overwrite changes merged
  in from another branch.

* Two git-annex processes can be doing conflicting things and inconsistent
  information be written to the journal.

  - One example would be concurrent get and drop of the same key.
    But could this really race? If the key was already present, the get
    would do nothing, so record no changes. If the key was not yet present,
    the drop would do nothing, and record no changes.

  - Instead, consider two copys of a key to different locations. If the
    slower copy starts first and ends last, it could cache the location
    info, add the  new location, and lose the other location it was copied to.
    Tested it and the location is not cached during the whole copy (logChange
    reads the current log immediatly before writing), so this
    race's window is very small -- but does exist.

----

## Updated plan

Make Branch.change transactional, so it takes a lock, reads a file,
applies a function to it, and writes the changed file.

Make Branch.update hold the same lock.

> [[Done]].


code:
On one of my ssh remotes I had to install git-annex using cabal. No system-wide installation possible. Hence `git-annex` and `git-annex-shell` are not in the default `$PATH` but in `$HOME/.cabal/bin`.

Right now the command run by git-annex when ssh'ing to a remote is hardcoded to "`git-annex-shell`", which doesn't work for me. It would be nice to be able to change this per remote, for example with an option named `annex.<remote>.annex-shell-command`. Changing "`git-annex-shell`" in `Remote/Helper/Ssh.hs` to "`~/.cabal/bin/git-annex-shell`" worked for me, but it's obviously very ugly :)

Could you do that please? I'll try to hack it myself and send you a patch in the next few days, but I'm pretty new to Haskell so it may take me a while... Thanks!

> [[closing|done]], see comments --[[Joey]]


code:
[[!comment format=mdwn
 username="http://joey.kitenet.net/"
 nickname="joey"
 subject="comment 1"
 date="2012-03-22T22:26:02Z"
 content="""
It doesn't need to be installed into the system PATH; just the user PATH. Which you should be able to control.

Exactly how to do this surely varies, but here I have a `~/.bashrc` containing `PATH=$HOME/bin:$PATH; export PATH` and I keep git-annex-shell in bin and it's available to eg \"ssh mybox git-annex-shell\"
"""]]


code:
[[!comment format=mdwn
 username="http://schnouki.net/"
 nickname="Schnouki"
 subject="comment 2"
 date="2012-03-23T13:27:12Z"
 content="""
Hmm, ok, solved. I'm using zsh, which is a little different: `.zshrc` is only read for interactive shells, so `ssh mybox 'echo $PATH'` displayed `/usr/bin:/bin:/usr/sbin:/sbin`. Using `.zshenv`, which is used even for non-interactive shells, did the trick. Thanks!
"""]]


code:
[[!comment format=mdwn
 username="carlo"
 ip="130.102.158.18"
 subject="comment 3"
 date="2013-07-01T04:20:22Z"
 content="""
For anyone who comes across this, make sure that you modify your $PATH before these lines in ~/.bashrc:

    # If not running interactively, don't do anything
    [ -z \"$PS1\" ] && return

"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="209.250.56.176"
 subject="comment 5"
 date="2014-05-27T16:13:09Z"
 content="""
See [[devblog/day_166__catching_up]]
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawmZilYULa6CDEGfuagoDlesyakBgnf-dF8"
 nickname="Maarten"
 subject="comment 5"
 date="2014-05-26T11:52:27Z"
 content="""
when doing ssh host 'echo $PATH', the remote shell running the command is not an interactive shell and thus does NOT source .bashrc, neither is it a login shell and as such does not source .profile or .bash_profile.  Which leaves me wondering: how does git annex log in to the remote machine to run git-annex-shell?  Does it do so via an interactive shell, a login shell, or neither?  Because if it does it via ssh host 'git-annex-shell', then neither .bashrc or .profile can configure its PATH.
"""]]


code:
This tag is for bugs that have been confirmed to be real bugs, and so are likely
to be the next bugs fixed.

If your bug report is not tagged as confirmed, you probably need to provide more
information so that the bug can be reproduced.


code:
I have a large direct-mode repository whose files I'm trying to copy to a non-direct-mode repository.  Both repositories live on an HDD attached to an rpi.

When I do $ git annex copy --to pi dirs/to/copy, the copy starts out OK, but eventually many files fail to copy.  The only diagnostic I get is "failed".  Judging from the backscroll, I don't see a strong pattern to the files which fail to copy; they're kind of interspersed amongst files which were successfully copied.  If I try to copy one of these failed files explicitly (git annex copy --to pi file/which/failed), this succeeds.  I have plenty of free space on the disk.

Is there a way to get more diagnostics out of git annex so I can see why these files are failing to copy?

> [[fixed|done]] --[[Joey]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawmUJBh1lYmvfCCiGr3yrdx-QhuLCSRnU5c"
 nickname="Justin"
 subject="comment 10"
 date="2014-03-06T21:38:53Z"
 content="""
Thanks a lot, Joey.

Compiling all of the dependencies for git-annex was taking forever on my pi, so I'll probably wait until the next release to test this out.  But I'll report back here if I have any problems.
"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="108.236.230.124"
 subject="comment 11"
 date="2014-03-10T19:20:42Z"
 content="""
Well you can always grab one of the daily builds, which include this fix.
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawmUJBh1lYmvfCCiGr3yrdx-QhuLCSRnU5c"
 nickname="Justin"
 subject="comment 1"
 date="2014-03-05T16:11:27Z"
 content="""
I tried git annex sync --content, and it failed to copy some files with

  git: createProcess: resource exhausted (Resource temporarily unavailable)

So this sounds like fork is failing; I'm probably exhausting my poor pi's RAM.  Maybe the same thing is happening for git annex copy.  I'll run strace to see.
"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="209.250.56.146"
 subject="comment 2"
 date="2014-03-05T20:31:57Z"
 content="""
How many files copied are we talking about before it begins to fail?

You can try passing --debug, which will make git-annex show every external command it runs, which includes `cp` for a copy to another repo on the same machine.

Might also check memory usage in top during the run.
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawmUJBh1lYmvfCCiGr3yrdx-QhuLCSRnU5c"
 nickname="Justin"
 subject="comment 3"
 date="2014-03-06T18:21:53Z"
 content="""
> How many files copied are we talking about before it begins to fail?

Tens of thousands of files processed, but many of them were already on the other remote so didn't invoke cp (or anything else).  ~3300 invocations of cp.

I saved a log of ps aux, and, while the memory used by git annex remains relatively constant, I do observe /tons/ of zombie processes.  3300, actually.

I didn't check all of them, but all of the zombie pids I checked appear to have corresponded to this command:

    /home/pi/git-annex.linux/shimmed/git/git --git-dir=/home/pi/hdd/annex/.git --work-tree=/home/pi/hdd/annex cat-file --batch

Perhaps git annex is forgetting to reap this processes?
"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="209.250.56.146"
 subject="comment 4"
 date="2014-03-06T18:32:33Z"
 content="""
Old versions of git-annex have known bugs involving zombies. What version?
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawmUJBh1lYmvfCCiGr3yrdx-QhuLCSRnU5c"
 nickname="Justin"
 subject="comment 5"
 date="2014-03-06T18:35:00Z"
 content="""
5.20140221-g1a47f5f -- I just downloaded it a week or two ago.
"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="209.250.56.146"
 subject="comment 6"
 date="2014-03-06T18:38:43Z"
 content="""
Hmm, that version should only start git cat-file --batch a maximum of 10 times (if it is crashing for some reason), and appears to wait on the process if it does crash. And if not, should only start one.

I think you need to post some git-annex --debug output , to show when it's running this command.
"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="209.250.56.146"
 subject="comment 7"
 date="2014-03-06T18:40:26Z"
 content="""
Actually, NM, I have reproduced the bug.
"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="209.250.56.146"
 subject="comment 8"
 date="2014-03-06T20:21:16Z"
 content="""
Analysis: Remote.Git's onLocal calls Annex.new to make a new AnnexState for the local remote. This state is not cached, and is regenerated for each file. Since it runs a Annex.Branch check of the location log on the remote, it needs to start catFile, and since the state is not reused, a new CatFileHandle is allocated each time. I'm not sure, but there may have been a recent-ish change that caused the location log to get checked and so catfile to be run; the general inneficiency of making a new AnnexState each time is not new.

Fixing this by caching the AnnexState will not only fix the resource leak, but should speed up local to local copies significantly!
"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="209.250.56.146"
 subject="comment 9"
 date="2014-03-06T21:17:14Z"
 content="""
Fixed in git. Also reduced the non-data-transfer work done by `git-annex copy` by around 8%.

I'm going to move this thread to [[bugs]] so I can close it. ;)
"""]]


code:
### Please describe the problem.

When copying files to s3 using,

    git annex copy --quiet --to mys3 --fast

No information whatsoever is printed during upload when ran without `--quite` it prints a line for each file in the repo creating page after page of output for repos with thousands of file basically no way to tell which files got uploaded. Is it possible to have a verbosity level between quite and verbose that only reports progress on actual copy/move operations.

### What steps will reproduce the problem?

### What version of git-annex are you using? On what operating system?

### Please provide any additional information below.

[[!format sh """
# If you can, paste a complete transcript of the problem occurring here.
# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log

# End of transcript or log.
"""]]

[[!meta title="copy --fast --to remote should be quiet when nothing to do"]]

> [[fixed|done]]; Avoided the unnecessary output in this situation.
> --[[Joey]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="209.250.56.87"
 subject="comment 1"
 date="2013-12-11T15:31:34Z"
 content="""
So, `git annex copy --to remote` currently does an active check that the remote has each file. This is a bit inneficient, and I have never been 100% sure that's the best thing, but from time to time I see a situation where trusting the location log when doing a `cp` would not be ideal.

It would be easy to omit the output for files that the check finds on the remote, but I think it's good because it tells you git-annex is doing actual work. Otherwise, `git annex copy --to remote` might seem to take a long time for no reason.

Anyway, you can disable the active check, with --fast. And I agree that in this case it would be better to not print anything. I looked at the code and it would be a little tricky to do it.

There's a simple workaround: `git annex copy --to remote --not --in remote`
"""]]


code:
[[!comment format=mdwn
 username="https://me.yahoo.com/a/FHnTlSBo1eCGJRwueeKeB6.RCaPbGMPr5jxx8A--#ce0d8"
 nickname="Hamza"
 subject="comment 2"
 date="2013-12-11T16:18:27Z"
 content="""
Joey thanks for tip. This actually made copy operation faster than `copy --fast`, and only provides the info that I need.
"""]]


code:
[[!format sh """
greg@x200s:~/Documents$ git-annex unused
unused . (checking for unused data...) (checking annex/direct/master...) (checking synced/annex/direct/master...) (checking synology/master...)
  Some annexed data is no longer used by any files:
    NUMBER  KEY
    1       SHA256E-s16367--0b00ef0154c42a0bf94e5be8d92d8af27455d59794f26c33dfc39c09178521c9.pdf
    2       SHA256E-s84--b08e1c831863bb43c02158bd5e8f3f5416c3a5203d89fa94a22149460142c273.odt
    3       SHA256E-s84--ec4caae451180a29721f2b6667eec8ec80eaa724f0727cf99d2bb21bf9218e9d.odt
    ...
    88      SHA256E-s84--710d69bef61674b04974ac550d713e5928563b2a12b902b64fe451705b967452.doc
    89      SHA256E-s3830--1348d6248e35625da3e22f73d2a0017185bb5e1aa37f65bbca5dfcb3c7f53034
    90      SHA256E-s119822--7c1b53ab6402b8835473f0b5c326f3cc300ac9372be79694942c1efa4bcdc621.pdf
    91      SHA256E-s84--63b6188696795885ff6570a76a3a74799396787f7058cbcfd4a2c40b22982420.odt
  (To see where data was previously used, try: git log --stat -S'KEY')

  To remove unwanted data: git-annex dropunused NUMBER

ok
greg@x200s:~/Documents$ git-annex copy --unused --to synology
"""]]

Is that correct behavior? I would assume the last command would at least run through the 91 files and check my synology remote that they are there. Like this repo did:

[[!format sh """
$ git-annex unused
unused . (checking for unused data...) (checking master...) (checking 60justin/master...)
  Some annexed data is no longer used by any files:
    NUMBER  KEY
    1       SHA256E-s9390266--7ed16c9423b331dbe63bb3b4278b8c94a6754a07177c53fceb3b24e9610e8054.NEF
    2       SHA256E-s10435713--49cbfe8466eada2c3787c9a7e158a7dfb9845a0aa8ef862ed2578b59c889dc4d.NEF
    3       SHA256E-s9442044--85c314e318f643237df5e3adf7559e9bf268ee28f1f92d4102161865323ddeb6.NEF
    4       SHA256E-s290672--c5822c3ef16bd62b5752b2dace81182ce00d64bd4d2d994ba93e3cb94e645708.JPG
    5       SHA256E-s293288--30f1367fc326f7b053012818863151206f9e3ddeab3c3fc5b5c1c573d120d50a.JPG
    6       SHA256E-s3672986--be960f6dc247df2496f634f7d788bd4a180fe556230e2dafc23ebc8fc1f10af3.JPG
  (To see where data was previously used, try: git log --stat -S'KEY')

  To remove unwanted data: git-annex dropunused NUMBER

ok
$ git-annex copy --unused --to synology
copy SHA256E-s9390266--7ed16c9423b331dbe63bb3b4278b8c94a6754a07177c53fceb3b24e9610e8054.NEF (checking synology...) ok
copy SHA256E-s10435713--49cbfe8466eada2c3787c9a7e158a7dfb9845a0aa8ef862ed2578b59c889dc4d.NEF (checking synology...) ok
copy SHA256E-s9442044--85c314e318f643237df5e3adf7559e9bf268ee28f1f92d4102161865323ddeb6.NEF (checking synology...) ok
copy SHA256E-s290672--c5822c3ef16bd62b5752b2dace81182ce00d64bd4d2d994ba93e3cb94e645708.JPG (checking synology...) ok
copy SHA256E-s293288--30f1367fc326f7b053012818863151206f9e3ddeab3c3fc5b5c1c573d120d50a.JPG (checking synology...) ok
copy SHA256E-s3672986--be960f6dc247df2496f634f7d788bd4a180fe556230e2dafc23ebc8fc1f10af3.JPG (checking synology...) ok
$
"""]]

> [[fixed|done]] per my comment --[[Joey]]


code:
[[!comment format=mdwn
 username="http://grossmeier.net/"
 nickname="greg"
 subject="indirect vs direct"
 date="2014-02-20T06:25:31Z"
 content="""
Aha! The issue is that the first repo (the one not copying unused files to synology) was in direct mode. I switched it to indirect and not only are there now a lot more files listed in unused, but copy --unused is working as expected.

Should there be a warning in git-annex unused when in direct mode about this? I'm not exactly sure what is happening (not sure why the number of unused would go up from 91 to 296).
"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="209.250.56.172"
 subject="Could I get a "version", brethren?"
 date="2014-02-20T17:58:32Z"
 content="""
I think there are two bugs here. I just reproduced and fixed a bug that would caused `git annex unused` to sometimes not notice certain unused keys when in direct mode. That's why the number you were finding in direct and indirect was different.

That does not explain why `copy --unused` would not operate on unused files when in direct mode. A problem I have not managed to reproduce..

There was a recent change to the format of the .git/annex/unused.log, which temporarily broke reading it (fixed in 5.20140210). This could be some version skew problem, as while the new git-annex version can read the old log format, the old git-annex version will ignore the new log format.
"""]]


code:
[[!comment format=mdwn
 username="http://grossmeier.net/"
 nickname="greg"
 subject="comment 3"
 date="2014-02-20T19:31:56Z"
 content="""
heh, I *just* dist-upgraded this morning on the box that was showing the problem from git-annex_5.20140127_amd64.deb to git-annex_5.20140210_amd64.deb. So what you say is probably right (re unused.log).

The only other annex I have in direct mode right now is one that I also am using the standalone build with (version 5.20131224-g6ca5271 right now). It has unused content in it (in fact, it's the synology annex, where I'm moving all the unused data to). I can do testing there if needed (and since it's a standalone build, it's easy for me to switch around git-annex versions with symlinks). The only problem is that it is dog slow when running git-annex unused. :)
"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="209.250.56.172"
 subject="comment 4"
 date="2014-02-20T19:49:21Z"
 content="""
Well, if you had a newer git-annex than 5.20140127 temporarily in path, it would certianly explain it. I have tested --unused in direct mode and am not seeing any problems.
"""]]


code:
[[!comment format=mdwn
 username="http://grossmeier.net/"
 nickname="greg"
 subject="comment 5"
 date="2014-03-05T04:34:49Z"
 content="""
Still seeing it with version 5.20140227

git-annex unused shows 110 keys.

git-annex copy --unused --to $anyremote just returns very quickly without doing anything.

git-annex copy --all --to $anyremote does go through every key and checks.

git-annex copy --key=$some_key_listed_in_unused --to $anyremote also just returns quickly without checking.

I even untrusted the repository via vicfg and passing the --untrust=remote argument.

Again, a direct mode repo that is also assistant'd.
"""]]


code:
[[!comment format=mdwn
 username="http://grossmeier.net/"
 nickname="greg"
 subject="comment 6"
 date="2014-03-05T04:38:26Z"
 content="""
Also, in another non-direct/non-assistant'd repo, things work as expected.
"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="209.250.56.146"
 subject="comment 7"
 date="2014-03-05T17:08:19Z"
 content="""
The only reason copy --key would do nothing is if the key is not locally present in your repository.

Please check if the keys listed by `git annex unused` are present in `.git/annex/objects/`
"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="209.250.56.146"
 subject="analysis"
 date="2014-03-06T21:37:15Z"
 content="""
[[!format haskell \"\"\"
        {- In indirect mode, look for the key. In direct mode,
         - the inode cache file is only present when a key's content
         - is present, so can be used as a surrogate if the content
         - is not located in the annex directory. -}
\"\"\"]]

Seems that is wrong.

I think that comment was based on removeAnnex calling removeInodeCache, which it does do.
And that is, afaik, the only point in git-annex where content files are removed from the annex.

However, in direct mode, removeAnnex is not the only way to delete a key's content -- the user can simply delete the file!
(Or a disk error could put it in lost+found, or whatever.)

That leaves the inode cache file present. unused will then behave as you describe. Also, uninit throws an ugly warning message
due to getKeysPresent giving it bad data. The only other caller of getKeysPresent is info, which will also operate on bad data and so generate slightly wrong stats.

I'm leaning toward making getKeysPresent do a full check of the cache and map, checking that the work tree still contains a key's content. This will make it somewhat slower (by 2 file reads and a stat() per key). So it would make sense to make a variant that only lists keys with content present in .git/annex/objects/. That could be used by `unused`, since by definition unused keys cannot have their content located in the work tree, so must have it in the object directory. uninit could also use it, since it's only interested in cleaning out .git/annex/objects. Only `info` will be slowed down.
"""]]


code:
### Please describe the problem.

This is a one-off thing, not a reproducible bug.  It was just so weird that I wanted to make a note of it in case somebody else had the same problem and it was reproducible.

I have a remote on the a USB drive which I mount once in a while on /Volumes/TOSHIBAEXT which has a remote on /Volumes/TOSHIBAEXT/annex.   It's a remote which I created by hand using "git clone" a long time ago, not something the assistant set up for me.

Anyway, today the assistant kept reporting that it could not sync to it.  I didn't know why, because I checked in my Finder sidebar and it showed a mounted drive called TOSHIBAEXT.  It's only when I checked with the command line that I noticed something was up!  There was now an ordinary directory at /Volumes/TOSHIBAEXT/annex.   This had apparently been created at some point when the drive was unmounted, so when it re-mounted, because there was a directory in the way, it re-mounted at "/Volumes/TOSHIBAEXT 1".  But it was displayed in the finder sidebar as just "TOSHIBAEXT" anyway because the finder hides this workaround from the user for whatever reason.

Presumably git-annex assistant at some point, for some reason, created an ordinary directory where it expected to find the annex.

I wonder if this may have to do with the fact that this is a non-bare, created-by-hand repo, from before I was using the assistant, not a normal bare remote that the assistant would have created for me if I'd been using the webapp.

### What steps will reproduce the problem?

I've only seen it once, and report the bug not as an outstanding issue but only as a heads-up that this has ever happened.

### What version of git-annex are you using? On what operating system?

git-annex version: 4.20130501-gb61740e

OS X lion (10.7)

### Please provide any additional information below.

I wasn't logging when this happened.

Again, just a heads-up; I'll keep my eye open for this happening again and post more info if it does.

[[!tag confirmed]]


code:
[[!comment format=mdwn
 username="http://edheil.wordpress.com/"
 ip="99.54.57.201"
 subject="comment 1"
 date="2013-05-05T20:55:19Z"
 content="""
Huh.  This happened a second time today.  I'll try and catch it in the act, and make a proper bug report, when I get a chance.

Still not sure this is a bug rather than just a peculiarity of how I have things set up.

"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 nickname="joey"
 subject="comment 2"
 date="2013-05-06T14:50:01Z"
 content="""
Hmm.. One way I can think that this could happen is if git-annex was running, and doing something in the repository on the drive that involved making a directory, and the drive was removed. There are several places in git-annex where it has code, like this:

<pre>
createAnnexDirectory
-- some action here that expects to have the git-annex directory
</pre>

Is the repo you have on the drive a direct mode repo by any chance? This is the only obvious way I can see that would cause it to create just the top level directory of the repository, and not a deeper directory tree like `.git/annex/tmp/`

The assistant also has a MountWatcher that detects when repositories that were not accessible get mounted, but it does not do anything to stop the repository being used when its drive gets unmounted. Even if it did, it couldn't go anything about code that is already running using the previously mounted repository. However, as things stand, it would probably also be possible for the drive to be removed, and some time to pass before an action was run that tried to do something to the no longer present repository.
So making the mountwatcher disable repositories when drives are unmounted would at least make this window narrower.
"""]]


code:
[[!comment format=mdwn
 username="http://edheil.wordpress.com/"
 ip="173.162.44.162"
 subject="comment 3"
 date="2013-05-06T16:52:07Z"
 content="""
It's possible, even likely, that there were indeed more directories and I didn't notice because they were hidden and I just looked at them with a vanilla \"ls\" command with no flags like \"-a\".

So it's probably exactly what you're thinking --t here was something going on, a transfer queued or something, that I didn't realize was happening and I removed it and it went and created the directory as if it were there.
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawkfHTPsiAcHEEN7Xl7WxiZmYq-vX7azxFY"
 nickname="Vincent"
 subject="comment 4"
 date="2013-07-24T14:19:01Z"
 content="""
I saw this too, today. The repo in my case was created via the webapp, using the 'usb key' option.
I was messing about with deleting repos and had turned off synchronisation for the repo on the key as I didn't have it inserted.

I only just stumbled on this bug report after removing the repository from the key entirely, so it's difficult to define the reproduction steps.

* create a 'usb key' repo via webapp, transfer type
* let it sync
* disable sync
* change a file in another repo, so there is something to sync
* unmount key, unplug key
* turn on sync again - should see 'failed ot sync with ...' in webapp dashboard
* reinsert key
* let it sync
* unplug key without properly unmounting
* change a file in another repo, so there is something to sync

I just did this. The key was mounting at /Volumes/VERBATIM4. Just after unplugging without unmounting, that directory was gone.
When I made the change that could be synced, I got a /Volumes/VERBATIM4 directory. The directory tree structure is similar to but different
from the structure on the usb key, see below.

Now when I try to plug it in again os/x will spot the potential conflict and mount the key at /Volumes/VERBATIM4\ 1.

If I instead rm -rf /Volumes/VERBATIM4 and then replug the usb key, everything syncs as expected.

Version: 4.20130723-ge023649 (os/x 10.8)

Build flags: Assistant Webapp Pairing Testsuite S3 WebDAV FsEvents XMPP DNS

Tree structure of the phantom directory:

    /Volumes/VERBATIM4/annex/
    └── annex
        ├── journal.lck
        ├── objects
        │   └── df3
        │       └── 043
        │           └── SHA256E-s17363--4c5ef74b4fcb9ecd962c6ecac694f87277e580836335e57924654762668a5448
        │               └── SHA256E-s17363--4c5ef74b4fcb9ecd962c6ecac694f87277e580836335e57924654762668a5448
        ├── tmp
        └── transfer
            ├── download
            │   └── adf30a67-777a-45a6-a658-e2266770f01b
            └── failed
                └── download
                    └── adf30a67-777a-45a6-a658-e2266770f01b
                        └── SHA256E-s17363--4c5ef74b4fcb9ecd962c6ecac694f87277e580836335e57924654762668a5448

    12 directories, 3 files

"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawmTNrhkVQ26GBLaLD5-zNuEiR8syTj4mI8"
 nickname="Juan"
 subject="Same happening here"
 date="2014-07-09T14:45:32Z"
 content="""
Any hint about how to stop this from happening again? This time I think I was careful and tried to stop git-annex, however I got ghosts.
Whats the correct way to unplug the drive?

My scenario is the following, I have two repos on HDD synced with a USB drive. I want to sync them to the USB drive from time to time when I plug it. I want a safe way to unplug drive without getting these ghost directories.

This time, I entered both directories in the HDD and made a git-annex assistant --stop, but after doing a ps aux | grep git-annex I still see many processes running so I'm not sure if I truly stopped it.
The webapp gives me the impression of working on a per-repo basis, so I miss having some sort of general feedback about git-annex. Something like: \"at the moment you are syncing repos A and B\" or \"idle\".
So I never have a clue about whats going on.
I'm trying to set up this since long time, and I'm a very capable unix guy, and I still have the feeling that I can't trust git-annex.
I really like the tool but I'm not sure how to use it properly.
Thanks in advance.

"""]]


code:
What steps will reproduce the problem?

I was trying to add a remote server repository. Unfortunately, this didn't work. Enter Host name, user name, directory, port
(left most of them at their default), but there is no way, to specify a password.) Clicking check this server failed:

 Failed to ssh to the server. Transcript: Permission denied, please try again. Permission denied, please try again. Permission denied (publickey,password).

(Problem was, I could never enter a password. Interestingly, on the konsole, I get a prompt for a password, but I can't enter anything there).

What is the expected output? What do you see instead?

Successfully create a connection and use the remote server.

What version of git-annex are you using? On what operating system?
Version: 3.20130124

Please provide any additional information below.

[[!tag /design/assistant done]]
[[!meta title="ssh password prompting issue with assistant"]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="4.154.3.125"
 subject="comment 1"
 date="2013-02-05T19:22:04Z"
 content="""
You're not really intended to start the git-annex assistant from a console. If you do, ssh will go ahead and prompt for passwords using that controlling console.

When the assistant is not started from a console, ssh should use the `ssh-askpass` to prompt for the password. Assuming your system has that installed.
"""]]


code:
[[!comment format=mdwn
 username="http://edward.myopenid.com/"
 nickname="edward"
 subject="Same problem on Android 4.3"
 date="2013-12-01T08:47:06Z"
 content="""
I have the same error on Android. Is git-annex on Android meant to come with ssh-askpass?
"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="209.250.56.64"
 subject="comment 3"
 date="2013-12-01T20:10:02Z"
 content="""
The bug your are commenting on is over a year old, and is closed, and does not involve Android. If you're having trouble with android, file a new bug report.

On android, git-annex runs from a terminal app. The ssh password prompting is currently done inside that app.
"""]]


code:
I just compiled ff7810eb83d8372e6206d487c63482d678e0b3d4 and created a new git-annex repository through the setup steps of "git-annex webapp". Then I tried configuring a Jabber account from the webapp. It then failed to create $REPO/.git/annex/creds/xmpp with a "No such file or directory" message because $REPO/.git/annex/creds did not get created. After doing a manual mkdir the Jabber setup went through fine.

> [[Fixed|done]], thanks. --[[Joey]]


code:
### Please describe the problem.

When `git annex direct` is interrupted (either through a power outage or deliberate `control-c`) it may leave the repository in an inconsistent state.

A typical situation is `git-annex` believing that the repo is in `indirect` mode while the files are not symlinks anymore.

I believe I have described this problem here before, but the bug report was deleted as part of the may 29th purge (222f78e9eadd3d2cc40ec94ab22241823a7d50d9,  [[bugs/git_annex_indirect_can_fail_catastrophically]]).

### What steps will reproduce the problem?

`git annex direct` on a large repository, `control-c` before it finishes.

Observe how a lot of files are now considered to be in the famous [[typechange status|forum/git-status_typechange_in_direct_mode/]] in git.

### What version of git-annex are you using? On what operating system?

5.20140717 on Debian Jessie, ext4 filesystem.

### Please provide any additional information below.

I wish i could resume the `git annex direct` command, but this will do a `git commit -a` and therefore commit all those files to git directly. It still seems to me that `git annex` should never run `git commit -a` for exactly that kind of situations.

I think that's it for now. -- [[anarcat]]

Update: i was able to get rid of the `typechange` situation by running `git annex lock` on the repository, but then all files are found to be missing by `git annex fsck`:

[[!format txt """
fsck films/God Hates Cartoons/VIDEO_TS/VTS_15_0.BUP (fixing location log)
  ** Based on the location log, films/God Hates Cartoons/VIDEO_TS/VTS_15_0.BUP
  ** was expected to be present, but its content is missing.

  Only 1 of 2 trustworthy copies exist of films/God Hates Cartoons/VIDEO_TS/VTS_15_0.BUP
  Back it up with git-annex copy.
"""]]

Oddly enough, the repo still uses hundreds of gigs, because all the files ended up in `.git/annex/misctmp`. Not sure I remember what happened there.

Similar issues and discussions:

* [[bugs/direct_mode_merge_interrupt/]]
* [[forum/Cleaning_up_after_aborted_sync_in_direct_mode/]]
* [[bugs/failure_to_return_to_indirect_mode_on_usb/]]
* [[forum/git-status_typechange_in_direct_mode/]]


code:
I ran the assistant in a subdir in direct mode, and it seemed to move files from other places outside that subdir
into it, and commit them there. These may have been files that needed to be committed, and it just staged them to the wrong place.

I'm pretty sure this does not affect indirect mode.

--[[Joey]]

The relevant commit, in my family's annex is
22e694549d698922389deb017c39c2b40371cdf0 --[[Joey]]

Was able to reproduce this as follows:

1. Make 2 repositories A and B, with B in direct mode.
2. In A, touch topfile; git annex add topfile; git annex sync
3. In B: mkdir subdir; cd subdir; git annex assistant

Result is a subdir/topfile appearing in B, which is wrong.

<pre>
 subdir/topfile | 1 +
 topfile   | 1 -
</pre>

Note that manually doing a git-annex sync in B's subdir does not cause this
to happen. It's specific to the assistant somehow.

--[[Joey]]

Before 2407170eaf78d9aa38d73d1af68c20da9882779b, git-annex sync in a subdir
had the same problem. That made it always run from the top of the work
tree, which its code actually assumes is the case.

The assistant, however, does not do that, and it's useful in general to
only run it in a subdir.

> Made sync merge code handle this correctly. Conflicted merge handing code
> was already ok. [[fixed|done]] --[[Joey]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="4.152.108.145"
 subject="comment 1"
 date="2013-08-03T20:15:17Z"
 content="""
Only seems to affect initial startup scan; files added outside the subdir after it was started up were not committed (which is the desired behavior when running the webapp in a subdir).
"""]]


code:
### Please describe the problem.

Running `git annex direct` in a repository may results with the following error message:

    git-annex: /home/mildred/Music/.git/annex/objects/2K/49/SHA256-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855/SHA256-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855.map898.tmp: rename: permission denied (Permission denied)
failed
    git-annex: direct: 1 failed

The major problem is that git-annex doesn't roll back the changes it did for the files it could successfully put in direct mode. Running git status show many files with typechange. The solution was to run `git add` on those files (although the hashing backend changed, so a commit must be created)

### What steps will reproduce the problem?

Don't know yet why the rename failed, but the direct mode should be rolled back if there is a problem. Restarting `git-annex direct` didn't result in an error.

### What version of git-annex are you using? On what operating system?

git-annex 5.20140405-g8729abc
arch-linux Linux moiraine 3.15.3-1-ARCH #1 SMP PREEMPT Tue Jul 1 07:32:45 CEST 2014 x86_64 GNU/Linux

### Please provide any additional information below.

[[!format sh """

$ git annex direct
commit
(Recording state in git...)
On branch master
nothing to commit, working directory clean
ok
direct .gitrefs/heads/annex/direct/master ok
direct .gitrefs/heads/git-annex ok
direct .gitrefs/heads/master ok
direct .gitrefs/heads/synced/master ok
direct .gitrefs/remotes/ashley/git-annex ok
direct .gitrefs/remotes/ashley/master ok
direct .gitrefs/remotes/ashley/synced/git-annex ok
direct .gitrefs/remotes/ashley/synced/master ok
direct .gitrefs/remotes/kylae/git-annex ok
direct .gitrefs/remotes/kylae/master ok
direct .gitrefs/remotes/kylae/synced/git-annex ok
direct ... ok
direct ... ok
direct ... ok
  /home/mildred/Music/.git/annex/objects/2K/49/SHA256-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855/SHA256-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855.map897.tmp: rename: permission denied (Permission denied)

  leaving this file as-is; correct this problem and run git annex fsck on it
direct ... ok
direct ... ok
direct ... ok
direct ... ok
direct ... ok

git-annex: /home/mildred/Music/.git/annex/objects/2K/49/SHA256-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855/SHA256-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855.map898.tmp: rename: permission denied (Permission denied)
failed
git-annex: direct: 1 failed

"""]]

[[!tag moreinfo]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="209.250.56.2"
 subject="comment 1"
 date="2014-07-10T18:11:22Z"
 content="""
The most likely problem would be if your repository contained annexed objects owned by different user than the one running `git annex direct`.

However, I cannot reproduce this problem:

<pre>
direct foo
  /home/joey/tmp/r/.git/annex/objects/pV/7j/SHA256E-s30--2754b7f82f6994005b97256273756f14d4abc17165c8819c06c07340d03351fa: setFileMode: permission denied (Operation not permitted)

  leaving this file as-is; correct this problem and run git annex fsck on it
direct  ok
</pre>

Since version 4.20130921, any exception when moving a file to direct mode should be caught like that.

I will need more information to reproduce your bug. Or are you sure you wrote down the right version of git-annex?
"""]]


code:
Direct mode merge handles the case where there's a conflict between local and remote files, that are checked into git.

However, the the local file (or directory, or symlink, whatever)
is not checked into git, the merge will overwrite it with the remote file from git.

> That's fixed; now this is detected and the local variant
> is renamed with ".variant-local", and possibly a number to make it
> unique.

New problem: If the merge pulls in a directory, and a file exists with
the name of the directory, locally, not annexed, the file is left alone,
but the directory is thus not checked out, and will be deleted on commit.

> [[fixed|done]]; regression test in place. --[[Joey]]


code:
Seems to me there is a bug in how merges are done in direct mode. This is
done in two steps:

1. Merge the remote branch into the local branch, with work tree directed
   to a temp dir.
2. Use the temp dir and the newly merged branch to update the work tree.

If this is interrupted between 1 and 2, by eg the user ctrl-Cing or power
being lost, the result is a repository that thinks the current branch has
been merged, but does not have an updated work tree. The next sync in that
repository will see the files as deleted (or as being an old version), and
commit the current work tree state to the branch.

Result is files appear to be lost, although `git revert` in an indirect
mode repo can get them back.

To fix this, direct mode merge would need to avoid updating the current
branch when merging the remote branch into it (how?). It should first
update the whole work tree, and only after it's updated should it update
the index and the current branch to reflect the merge.

This way, if the merge is interrupted, the work tree may have uncommitted
changed -- but it's fine if they get accidentially committed, since when
the merge is re-done, those changes will by the same ones made by the
merge. (I assume this is how `git merge` normally works.) --[[Joey]]

> Implemented that. And then realized that even updating the index
> as part of a merge results in the work tree being out of sync with the
> index. Which will cause the next sync to again delete any files that
> are in the index but not the work tree. Urgh.
>
> Seems that a direct mode
> merge also needs to use a different index file to stage its changes?
> (Ugh)
> > done --[[Joey]]

> > > I had to revert the fix on FAT/Windows due to
> > > a git bug: <http://marc.info/?l=git&m=140262402204212&w=2>
> > > Once that bug's fixed, I can revisit this. --[[Joey]]

[[!meta title="direct mode merge interrupt (fixed for all except FAT, Windows)"]]

## other options

> Or could perhaps use `git-merge-tree`
> and avoid staging the merge in the index until the work-tree is updated.
>
> Alternatively, could use another strategy.. Add a lock file which is held while
> the merge is in progress and contains the pre-merge sha.
> If the lock file is present but not held, state is inconsistent.
> `git-annex sync` and the SanityChecker should
> then run mergeDirectCleanup to recover, before any commits can be made
> from the inconsistent state. This approach seems to get complicated
> quickly.. --[[Joey]]


code:
Some filesystems have stupid rules about characters not allowed in filenames. For example, FAT doesn't allow '?' '*' ':' etc.

The direct mode merge code lets `git merge` update a temp directory with the new files from the merge, before doing its work tree update and committing. This can fail:

error: unable to create file non-rus/Dance/Dream_Dance_Vol15/CD1/09-??.mp3 (Invalid argument)

This leaves the work tree without the file, and the index knows about the file. Result is that the next time a commit is done, this file appears to have been deleted, and that is committed and propigates out. Which can be surprising.

----

It would probably be better if, when the working tree cannot be updated, it left the repository in some state that would not make the next commit remove anything.

Ie, direct mode should replicate this behavior:

root@darkstar:/home/joey/mnt>git init
root@darkstar:/home/joey/mnt>git merge FETCH_HEAD
error: unable to create file foo? (Invalid argument)
fatal: read-tree failed
root@darkstar:/home/joey/mnt>git status
On branch master

Initial commit

nothing to commit (create/copy files and use "git add" to track)

Problem is, the call to `git merge` can also fail due to a conflict. In that case, git-annex wants to continue with automatic conflict resolution.
So, how to detect when `git merge` has skipped creating illegal filenames?

----

Alternatively, git-annex could learn/probe the full set of characters not allowed in filenames, and examine merges before performing them, and refuse to do anything if the merge added an illegal filename.a

[[!tag confirmed]]


code:
### Please describe the problem.

git-annex attempts to create files with question marks in them on my direct repository on a FAT32 drive.  This fails.

### What steps will reproduce the problem?

git annex addurl 'http://rf.proxycast.org/m/media/273073201426.mp3?c=culture&p=La+t%C3%AAte+au+carr%C3%A9_10212&l3=20131002&l4=&media_url=http%3A%2F%2Fmedia.radiofrance-podcast.net%2Fpodcast09%2F10212-02.10.2013-ITEMA_20531825-0.mp3'

### What version of git-annex are you using? On what operating system?

[[!format sh """
$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=12.04
DISTRIB_CODENAME=precise
DISTRIB_DESCRIPTION="Ubuntu 12.04.3 LTS"

$ git annex version
git-annex version: 4.20130922-g7dc188a
build flags: Assistant Webapp Pairing Testsuite S3 WebDAV Inotify DBus XMPP Feeds Quvi
local repository version: 4
default repository version: 3
supported repository versions: 3 4
upgrade supported from repository versions: 0 1 2
"""]]

### Please provide any additional information below.

[[!format sh """
# If you can, paste a complete transcript of the problem occurring here.
# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log

$ git annex addurl 'http://rf.proxycast.org/m/media/273073201426.mp3?c=culture&p=La+t%C3%AAte+au+carr%C3%A9_10212&l3=20131002&l4=&media_url=http%3A%2F%2Fmedia.radiofrance-podcast.net%2Fpodcast09%2F10212-02.10.2013-ITEMA_20531825-0.mp3'
addurl rf.proxycast.org_m_media_273073201426.mp3_c=culture&p=La+t%C3%AAte+au+carr%C3%A9_10212&l3=20131002&l4=&media_url=http%3A%2F%2Fmedia.radiofrance-podcast.net%2Fpodcast09%2F10212-02.10.2013-ITEMA_20531825-0.mp3 (downloading http://rf.proxycast.org/m/media/273073201426.mp3?c=culture&p=La+t%C3%AAte+au+carr%C3%A9_10212&l3=20131002&l4=&media_url=http%3A%2F%2Fmedia.radiofrance-podcast.net%2Fpodcast09%2F10212-02.10.2013-ITEMA_20531825-0.mp3 ...)
/media/sixtyfour/audio/.git/annex/tmp/URL--http&c%%rf.proxycast.org%m%media%273073201426.mp3?c=culture&ap=La+t&sC3&sAAte+au+carr&sC3&sA9_10212&al3=20131002&al4=&amedia_url=http&s3A-86e83c2aaa925b3bc337c1d6a27bd300: Argument invalide

git-annex: /media/sixtyfour/audio/.git/annex/transfer/failed/download/00000000-0000-0000-0000-000000000001/URL--http&c%%rf.proxycast.org%m%media%273073201426.mp3?c=culture&ap=La+t&sC3&sAAte+au+carr&sC3&sA9_10212&al3=20131002&al4=&amedia_url=http&s3A-86e83c2aaa925b3bc337c1d6a27bd300: openFile: invalid argument (Invalid argument)
failed
git-annex: addurl: 1 failed

# End of transcript or log.
"""]]

> [[fixed|done]] --[[Joey]]


code:
recently fixed [[bugs]]

[[!inline pages="./* and link(./done) and !*/Discussion" sort=mtime show=10
archive=yes]]


code:
I was trying out the example with the walkthrough using_the_URL_backend. I tried dropping files that I had after doing an "git annex get ." which have the URL backend associated with the files it fails with

<pre>
[jtang@lenny gc]$ git annex drop -v curl-7.21.4.tar.gz
drop curl-7.21.4.tar.gz
failed
git-annex: 1 failed
</pre>

At first I thought it was just my OSX machine not having the coreutils stuff load up before the BSD utils, but I then tried the same thing on my archlinux machine and it showed the same behaviour, that is I could not drop a file with the URL backend as shown in the walkthrough.

> Whoops, got some logic backwards. [[fixed|done]]! --[[Joey]]


code:
### Please describe the problem.

I tried to create an annex (``git annex init foo``) inside a ``encfs`` mount, and git-annex says that it's crippled, and disables core.symlinks and goes into direct mode

### What steps will reproduce the problem?

    apt-get install encfs
    encfs -o kernel_cache empty_dir other_empty_dir
    cd other_empty_dir
    git init
    git annex init foo

### Expected results

    init foo ok
    (Recording state in git...)

### Actual results

    init foo
      Detected a crippled filesystem.

      Disabling core.symlinks.

      Enabling direct mode.
    ok
    (Recording state in git...)

### What version of git-annex are you using? On what operating system?

4.20130601 on debian unstable

### P.S.

This was particularly annoying when I tried this on a bare repository. I'm pretty sure bare repositories don't need symlinks, and should definitely not be in direct mode. Hopefully you can fix it before I have time to file another bug report :)

Thank you!

> [[fixed|done]] --[[Joey]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 nickname="joey"
 subject="comment 1"
 date="2013-06-10T17:22:11Z"
 content="""
encfs appears to not allow creation of hard links to files:

[[!format sh \"\"\"
joey@gnu:~/tmp/other_empty_dir>ln me me2
ln: failed to create hard link `me2' => `me': Operation not permitted
\"\"\"]]

According to the documentation, this is a limitation when using External IV Chaining. And only a problem if you choose paranoia mode when setting up encfs.

git-annex uses hard links in a limited number of code paths, most notably `git annex add` uses them when locking down files.

encfs seems like something I'd like to interoperate well with git-annex. So, it seems worthwhile to break out lack of hard links support from the other limitations currently lumped into \"cripped file system\". I've done so.

----

Also made it stop setting direct mode on bare repositories, although that configuration had no effect anyway.
"""]]


code:
### Please describe the problem.
Could not get ssh-askpass running on macosx.
Transfered the publich key with scp.
certificate based ssh from macosx  to ssh server (debian testing) works.
After successfull login to ssh server git annex stops with the following errors:

Browser Error Message:
user error (gpg ["--batch","--no-tty","--use-agent","--quiet","--trust-model","always","--gen-random","--armor","1","512"] exited 2)

### What steps will reproduce the problem?
1. git annex on debian gnu linux
2. git annex on macosx
3. set up "share with a friend"
4. create rsa keys on macosx  "ssh-kegen -t rsa"
5. scp public key to server with hosts encrypted ssh remote
6. configure the server use a encrypted ssh remote in tranport mode

### What version of git-annex are you using? On what operating system?
current debian testing (20130827)
macosx 20130827
### Please provide any additional information below.

[[!format sh """
# If you can, paste a complete transcript of the problem occurring here.
# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log

(scanning...) [2013-09-28 17:39:25 CEST] Watcher: Performing startup scan
(started...) [2013-09-28 17:39:26 CEST] XMPPSendPack: Syncing with jlueters
Everything up-to-date
[2013-09-28 17:39:30 CEST] XMPPSendPack: Unable to download files from jlueters.

(encryption setup) gpg: /Users/lambert/.gnupg/gpg.conf:241: invalid auto-key-locate list
28/Sep/2013:17:40:06 +0200 [Error#yesod-core] user error (gpg ["--batch","--no-tty","--use-agent","--quiet","--trust-model","always","--gen-random","--armor","1","512"] exited 2) @(yesod-core-1.1.8.3:Yesod.Internal.Core ./Yesod/Internal/Core.hs:550:5)
(encryption setup) gpg: /Users/lambert/.gnupg/gpg.conf:241: invalid auto-key-locate list
28/Sep/2013:17:40:48 +0200 [Error#yesod-core] user error (gpg ["--batch","--no-tty","--use-agent","--quiet","--trust-model","always","--gen-random","--armor","1","512"] exited 2) @(yesod-core-1.1.8.3:Yesod.Internal.Core ./Yesod/Internal/Core.hs:550:5)

# End of transcript or log.
"""]]

> [[dup|done]] --[[Joey]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="4.153.8.80"
 subject="comment 1"
 date="2013-09-29T19:13:59Z"
 content="""
This is a duplicate of this bug report: [[Error_creating_encrypted_cloud_repository: \"internal_server_error\"]]

To work around, you need to edit ~/.gnupg/gpg.conf and remove or edit the `auto-key-locate` line.
"""]]


code:
`git annex get` becomes very unhappy as soon as `$PWD` is gone.
The easiest way is to `git checkout git-annex` while you are getting in another tab.

This results in git-annex entering an endless loop:

    fatal: Unable to read current working directory: No such file or directory
    fatal: Unable to read current working directory: No such file or directory
    fatal: Unable to read current working directory: No such file or directory
    fatal: Unable to read current working directory: No such file or directory
    fatal: Unable to read current working directory: No such file or directory
    fatal: Unable to read current working directory: No such file or directory
    fatal: Unable to read current working directory: No such file or directory
    fatal: Unable to read current working directory: No such file or directory
    fatal: Unable to read current working directory: No such file or directory

> [[fixed|done]] --[[Joey]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="209.250.56.227"
 subject="comment 1"
 date="2014-01-02T01:11:15Z"
 content="""
This is due to the code that restarts git hash-object if it crashes (because it has been known to crash before). Utility.CoProcess needs to have a limit to the number of times it retries when something is failing.
"""]]


code:
Hi,

My Webapp isn't working:

    $ git-annex webapp error: refs/gcrypt/gitception+ does not point to a valid object!
    error: refs/remotes/Beta/git-annex does not point to a valid object!
    error: refs/remotes/Beta/master does not point to a valid object!
    fatal: unable to read tree 656e7db5be172f01c0b6994d01f1a08d1273af12

So I tried to repair it:

    $ git-annex repair Running git fsck ...
    Stack space overflow: current size 8388608 bytes. Use `+RTS -Ksize -RTS' to increase it.

So I tried to follow your advice here and increase the stack:

    $ git-annex +RTS -K35000000 -RTS fsck
    git-annex: Most RTS options are disabled. Link with -rtsopts to enable them.

I wasn't sure what to do next, so any help would be appreciated.

> Now only 20k problem shas max (more likely 10k) are collected from fsck,
> so it won't use much memory (60 mb or so). If it had to truncate
> shas from fsck, it will re-run fsck after the repair process,
> which should either find no problems left (common when eg when all missing shas
> were able to be fetched from remotes), or find a new set of problem
> shas, which it can feed back through the repair process.
>
> If the repository is very large, this means more work, but it shouldn't
> run out of memory now. [[fixed|done]] --[[Joey]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="209.250.56.172"
 subject="comment 1"
 date="2014-02-23T18:51:45Z"
 content="""
I suspect that git fsck is outputting so many lines about problems that it's taking more memory than it's limited to using to hold them all.

Can you paste the output of: git fsck --no-dangling --no-reflogs
"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="209.250.56.172"
 subject="comment 2"
 date="2014-02-23T19:09:30Z"
 content="""
Erm, that output is liable to be big, I only care how many lines and characters of output there are!

    git fsck --no-dangling --no-reflogs |wc
"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="209.250.56.172"
 subject="comment 3"
 date="2014-02-23T19:12:10Z"
 content="""
Also, you can build git-annex from source with the RTS options enabled by running `cabal install git-annex --ghc-options=-rtsopts`

(or just build git-repair which has the repository repair parts of git-annex)
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawnFjuvfPpi1kf6l54bxfFUm0Aw_Gf_IO0o"
 nickname="Aaron"
 subject="Didn't work"
 date="2014-03-01T22:40:36Z"
 content="""
Many thanks for your input, Joey.

That didn't seem to work.

$ git fsck --no-dangling --no-reflogs |wc
error: unknown option `no-dangling'
usage: git fsck [options] [<object>...]

    -v, --verbose         be verbose
    --unreachable         show unreachable objects
    --tags                report tags
    --root                report root nodes
    --cache               make index objects head nodes
    --reflogs             make reflogs head nodes (default)
    --full                also consider packs and alternate objects
    --strict              enable more strict checking
    --lost-found          write dangling objects in .git/lost-found
    --progress            show progress

      0       0       0

$ git --version
git version 1.7.9.5
"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="209.250.56.146"
 subject="comment 5"
 date="2014-03-05T16:53:33Z"
 content="""
You have an older version of git, which does not support --no-dangling.

git-annex will detect that, and omit the option. This does make it more likely that git fsck is outputing a ton of information, so I continue to think that's the most likely cause of the memory use.

Please verify with: git fsck --no-reflogs |wc
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawnFjuvfPpi1kf6l54bxfFUm0Aw_Gf_IO0o"
 nickname="Aaron"
 subject="comment 6"
 date="2014-03-08T00:08:22Z"
 content="""
That seemed to work, thanks:

    $ git fsck --no-reflogs |wc
    Checking object directories: 100% (256/256), done.
    error: refs/gcrypt/gitception+ does not point to a valid object!
    error: refs/remotes/Beta/git-annex does not point to a valid object!
    error: refs/remotes/Beta/master does not point to a valid object!
    Checking connectivity: 128728, done.
     369082 1165340 20898546

Thanks for your help!
"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="108.236.230.124"
 subject="comment 7"
 date="2014-03-10T18:02:46Z"
 content="""
Ok, that verifies my hypothesis that fsck is outputting a lot of lines. 369082 lines to be precise, comprising 20 mb of data in all. So it's not too surprising this ends up blowing up into a bad amount of memory use.

I'm going to move this from a forum post over to a bug: <http://git-annex.branchable.com/bugs/enormous_fsck_output_OOM>
"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="108.236.230.124"
 subject="comment 8"
 date="2014-03-10T18:12:29Z"
 content="""
In a quick test with a 32 mb dummy fsck output, git-repair ballooned up to 1.7 gb. Clearly something not happy there, although in my case this did not cause it to crash.
"""]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="108.236.230.124"
 subject="comment 9"
 date="2014-03-10T19:17:34Z"
 content="""
I've gotten this down to 900 mb used, in the case where every single line lists a different sha. Possibly more important, if lines repeat shas, or are extraneous, memory usage will be significantly lower. This might be enough to get it working in Aaron's repository, especially if the bulk of the git fsck output was about dangling objects, which are now ignored without buffering them all in memory.

The memory usage is just about as low as is possible; it takes a fair amount of memory just to hold 300 thousand shas in memory. And the git repair process needs to keep track of every broken sha. (Maybe there's a way to stream them, but I don't immediately see one.)

I hesitate to say this means the problem is truly fixed. I have some much larger repositories with eg, `git count-objects -v` reporting 2 million objects. If they all went corrupt, it would still use too much memory.

One improvement would be to store Shas in packed memory, rather than as strings like they are now. That would probably half the memory used. It still does not seem like a full solution.
"""]]


code:
### Please describe the problem.
I'm not sure if you'll consider this a bug, or if I'm just doing something wrong, but I'm having trouble compiling git-annex on OmniOS (a derivative of OpenSolaris). I've got GHC 7.6.3 built and installed (bootstrapped using the Solaris binaries for GHC 7.0.3). I used it to build haskell-platform (although I had to disable the OpenGL-related packages to do so), and now I'm trying to use cabal to build git-annex and its dependencies. I've started with a minimal build; this will end up as an archive remote so it should be sufficient. The instructions <http://git-annex.branchable.com/install/cabal/> say to run

    cabal install git-annex --bindir=$HOME/bin -f"-assistant -webapp -webdav -pairing -xmpp -dns"

This builds a bunch of stuff but then fails to compile the network-info package. As I understand it, the git-annex package only needs network-info if it's compiled with pairing supoort (I'm looking at <https://github.com/joeyh/git-annex/blob/master/git-annex.cabal>), and this command is telling it to disable pairing.

Is there some other dependency that needs network-info? Is there a way to find out?

Thanks

> fowarded 2 ways; [[done]] --[[Joey]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="108.236.230.124"
 subject="comment 1"
 date="2014-06-06T16:53:21Z"
 content="""
http://packdeps.haskellers.com/reverse/network-info

It's uuid which needs network-info -- but not for the parts of uuid which git-annex uses. (git-annex does not use the uuids that include the MAC address).

It's not hard to avoid that dependency. The attached patch will do. I've forwarded this to the author of uuid.

<pre>
diff --git a/uuid.cabal b/uuid.cabal
index 0a53059..f00e887 100644
--- a/uuid.cabal
+++ b/uuid.cabal
@@ -24,6 +24,9 @@ Extra-Source-Files:
      CHANGES
      CONTRIBUTORS

+Flag NetworkInfo
+  Description: Enable UUIDs that need network-info
+  Default: True

 Library
  Build-Depends: base >=3 && < 5,
@@ -32,17 +35,20 @@ Library
                 cryptohash >= 0.7 && < 0.12,
                 deepseq == 1.3.*,
                 hashable (>= 1.1.1.0 && < 1.2.0) || (>= 1.2.1 && < 1.3),
-                network-info == 0.2.*,
                 random >= 1.0.1 && < 1.1,
                 time >= 1.1 && < 1.5
+ if flag(NetworkInfo)
+   Build-Depends: network-info == 0.2.*

  Exposed-Modules:
    Data.UUID
-   Data.UUID.Util
-   Data.UUID.V1
    Data.UUID.V3
    Data.UUID.V4
    Data.UUID.V5
+ if flag(NetworkInfo)
+   Exposed-Modules:
+     Data.UUID.Util
+     Data.UUID.V1

  Other-Modules:
    Data.UUID.Builder
</pre>
"""]]


code:
[[!comment format=mdwn
 username="https://www.google.com/accounts/o8/id?id=AItOawm5WyknJirJJridJjiPNgrlYxGG9xrZBvA"
 nickname="Daniel"
 subject="a different fix"
 date="2014-06-07T04:27:47Z"
 content="""
Thanks. I also went back and took a closer look at the error; it looks like fixing network-info is pretty easy as well. <https://github.com/jystic/network-info/pull/10>
"""]]


code:
**What steps will reproduce the problem?**

Delete the last repository in the current repository (where it says: warning type in "yes please do as i say").
Then try running git annex webapp

**What is the expected output? What do you see instead?**

Webapp starts. git-annex: Not in a git repository.

**What version of git-annex are you using? On what operating system?**

4.20130417. Debian wheezy/testing

**Please provide any additional information below.**

See also
[[bugs/git-annex:_Not_in_a_git_repository._/]]
where the same error message occurs

[[!tag /design/assistant moreinfo]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 nickname="joey"
 subject="comment 1"
 date="2013-04-23T16:39:08Z"
 content="""
Do you still have a `~/annex` or `~/Desktop/annex`? What's in there?

Also, please show me the content of your `~/.config/git-annex/autostart` file.

The only bug I have been able to track down is that it fails to remove the repository from the autostart file, and so if the repository directory still exists, this will happen. I've fixed it to properly remove the repository from the file, but hesitate to close this bug as I don't understand how the repository directory could still exist after it's supposed to be deleted.

(When I try, it does get deleted, and starting the webapp again allows starting over with a new repository.)
"""]]


code:
When using `git annex get foo` where foo is available in a rsync remote with encryption I got an error saying that rsync cannot
find the required file but extra ' are here.

I attached a patch for this.

> But you didn't, sadly. :(
>
> I don't seem to see the problem, set up a rsync over ssh with encryption
> and sent over a file "foo", and then got it back from rsync, without
> trouble.
>
> Ah, you're not using rsync over ssh, but just to a local directory,
> right? --[[Joey]]

>> [[fixed|done]] --[[Joey]]


code:
Two clients, both in direct mode, both running assistant.

When a change is made on A, the assistant commits it to annex/direct/master. But, the master branch is not changed.

B notices there is a change, pulls from A. Gets annex/direct/master, but does not merge it into its local branch at all.

[[!tag confirmed]]

> [[fixed|done]] --[[Joey]]


code:
[[!comment format=mdwn
 username="http://joeyh.name/"
 ip="108.236.230.124"
 subject="comment 1"
 date="2014-06-16T15:10:42Z"
 content="""
Reproduced this as described.

Only happens when using the assistant in the second repository, which is a ssh remote of the first, and when there is not also a remote going the other way (so does not affect pairing).

As I thought, the problem is that the master branch is not updated by the assistant when in direct mode, and while annex/direct/master is pulled by the other assistant, it does not merge it.

"""]]