Repository navigation
git.repo.base.repo.tree() blocks deletion of repo directory on Windows (fine in Linux) #546
Closed as not planned
Description
Activity
After the
tree()call there are two entire Git processes that are left open... Pausing in the debugger and killing those process trees manually (via Process Explorer) does allow clean deletion of the directories.The 'offending' instances of Git are launched per the below stack traces. I did snip the last few stack frames that were for the PyCharm debugger activities.
>>> import traceback; traceback.print_stack() File "C:\Program Files (x86)\JetBrains\PyCharm Community Edition 2016.2.3\helpers\pydev\pydevd.py", line 1580, in <module> globals = debugger.run(setup['file'], None, None, is_module) File "C:\Program Files (x86)\JetBrains\PyCharm Community Edition 2016.2.3\helpers\pydev\pydevd.py", line 964, in run pydev_imports.execfile(file, globals, locals) # execute the script File "C:\Program Files (x86)\JetBrains\PyCharm Community Edition 2016.2.3\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile exec(compile(contents+"\n", file, 'exec'), glob, loc) File "C:/Users/IEUser/Desktop/407/t.py", line 40, in <module> tree = repo.tree('master') File "c:\users\ieuser\desktop\407\gitpython\git\repo\base.py", line 442, in tree return self.rev_parse(text_type(rev) + "^{tree}") File "c:\users\ieuser\desktop\407\gitpython\git\repo\fun.py", line 193, in rev_parse obj = name_to_object(repo, rev[:start]) File "c:\users\ieuser\desktop\407\gitpython\git\repo\fun.py", line 130, in name_to_object return Object.new_from_sha(repo, hex_to_bin(hexsha)) File "c:\users\ieuser\desktop\407\gitpython\git\objects\base.py", line 64, in new_from_sha oinfo = repo.odb.info(sha1) File "c:\users\ieuser\desktop\407\gitpython\git\db.py", line 37, in info hexsha, typename, size = self._git.get_object_header(bin_to_hex(sha)) File "c:\users\ieuser\desktop\407\gitpython\git\cmd.py", line 930, in get_object_header cmd = self._get_persistent_cmd("cat_file_header", "cat_file", batch_check=True) File "c:\users\ieuser\desktop\407\gitpython\git\cmd.py", line 913, in _get_persistent_cmd cmd = self._call_process(cmd_name, *args, **options) File "c:\users\ieuser\desktop\407\gitpython\git\cmd.py", line 868, in _call_process return self.execute(call, **_kwargs) File "c:\users\ieuser\desktop\407\gitpython\git\cmd.py", line 586, in execute proc = Popen(command,
and
>>> import traceback; traceback.print_stack() File "C:\Program Files (x86)\JetBrains\PyCharm Community Edition 2016.2.3\helpers\pydev\pydevd.py", line 1580, in <module> globals = debugger.run(setup['file'], None, None, is_module) File "C:\Program Files (x86)\JetBrains\PyCharm Community Edition 2016.2.3\helpers\pydev\pydevd.py", line 964, in run pydev_imports.execfile(file, globals, locals) # execute the script File "C:\Program Files (x86)\JetBrains\PyCharm Community Edition 2016.2.3\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile exec(compile(contents+"\n", file, 'exec'), glob, loc) File "C:/Users/IEUser/Desktop/407/t.py", line 40, in <module> tree = repo.tree('master') File "c:\users\ieuser\desktop\407\gitpython\git\repo\base.py", line 442, in tree return self.rev_parse(text_type(rev) + "^{tree}") File "c:\users\ieuser\desktop\407\gitpython\git\repo\fun.py", line 216, in rev_parse obj = to_commit(obj).tree File "C:\Users\IEUser\Desktop\407\venv\lib\site-packages\gitdb\util.py", line 237, in __getattr__ self._set_cache_(attr) File "c:\users\ieuser\desktop\407\gitpython\git\objects\commit.py", line 143, in _set_cache_ binsha, typename, self.size, stream = self.repo.odb.stream(self.binsha) # @UnusedVariable File "c:\users\ieuser\desktop\407\gitpython\git\db.py", line 42, in stream hexsha, typename, size, stream = self._git.stream_object_data(bin_to_hex(sha)) File "c:\users\ieuser\desktop\407\gitpython\git\cmd.py", line 947, in stream_object_data cmd = self._get_persistent_cmd("cat_file_all", "cat_file", batch=True) File "c:\users\ieuser\desktop\407\gitpython\git\cmd.py", line 913, in _get_persistent_cmd cmd = self._call_process(cmd_name, *args, **options) File "c:\users\ieuser\desktop\407\gitpython\git\cmd.py", line 868, in _call_process return self.execute(call, **_kwargs) File "c:\users\ieuser\desktop\407\gitpython\git\cmd.py", line 586, in execute proc = Popen(command,
Does not seem to work (Still throws exception when deleting directory. I guess this just means it's not setup as a context manager):
with git.Repo.clone_from(url, dir) as repo: with repo.tree('master') as tree: pass
Does work (No exception when deleting directory):
repo = git.Repo.clone_from(url, dir) tree = repo.tree('master') del repo del tree
So, perhaps it is not a bug and I simply need to make sure that my repo and tree get deleted?
Better 'workaround':
tree = repo.tree('master') repo.git.clear_cache()
Reacted by Simon Schürrle and Garrett Brenner- added a commit that references this issue
on Oct 31, 2016 - added 3 commits that reference this issue
on Oct 7, 2021 - added a commit that references this issue
on Oct 8, 2021 - added 4 commits that reference this issue
on Jul 18, 2022 - added a commit that references this issue
on Jul 20, 2022 - OS Name: Microsoft Windows 10 Pro
- OS Version: 10.0.19045
- Python version: 3.10.9
Workaround
def main(): with git.Repo.clone_from(REPO, local_path) as repo: # omitted for brevity print(f"Removing temp destination directory {local_path}") git.rmtree(local_path)Reacted by Mauro and Sebastian ThielReacted by MauroI had the same problem just with creating a repo and deleting it again.
Thankfully, @gdyrrahitis solution worked for me 🚀I assume there was no progress on this issue?
I'm facing something which seems the same issue but even a
rmtreeand aclean_cachedoes not help.path = WindowsPath('C:/Users/XXXX/AppData/Local/Temp/tmp9__47p3j/template'), onexc = <function rmtree.<locals>.onexc at 0x0000 def _rmtree_unsafe(path, onexc): def onerror(err): onexc(os.scandir, err.filename, err) results = os.walk(path, topdown=False, onerror=onerror, followlinks=os._walk_symlinks_as_files) for dirpath, dirnames, filenames in results: for name in dirnames: fullname = os.path.join(dirpath, name) try: os.rmdir(fullname) except OSError as err: onexc(os.rmdir, fullname, err) for name in filenames: fullname = os.path.join(dirpath, name) try: > os.unlink(fullname) E PermissionError: [WinError 5] Zugriff verweigert: 'C:\\Users\\XXXX\\AppData\\Local\\Temp\\tmp9__47p3j\\temffb21d689bc799b9a573' ..\AppData\Local\Programs\Python\Python312\Lib\shutil.py:633: PermissionErrorcode snipped
repo.git.clear_cache() git.rmtree(git_repo_path / ".git") shutil.rmtree(tmpdir)I don't really had time to debug it more, and just went with a try, except and ignoring the cleanup.
I am checking out repositories into a temporary directory, doing various things with them including calling
tree(), and then deleting the directories and the temporary directory. This works fine in Linux but not in Windows. I have created this SSCCE to show the issue. I will take some time right now to try to attempt to fix it myself but I like to document my issues first (so others know and just in case someone else sees it and has insight).Note that I am making an effort (
del_rw()) to clear read-only items.I am testing on:
t.py.txt
Output: