Ticket #62 (closed defect: fixed)

Opened 5 years ago

Last modified 5 years ago

FTP upload path not correct using ftp_sync.py under win32

Reported by: ftputiluser Owned by: schwa
Priority: critical Milestone: 2.7
Component: Library Version:
Keywords: ftp_sync, sync, Windows Cc:

Description

The upload path is not correct under win32 when using ftp_sync.py For example:

dir/some.file

will be uploaded to FTP as 1 directory and 1 file named "dir\some.file"

dir/ dir\some.file

The bug is because of the default path seperator is "\" under win32. os.path.sep should be considered.

Attachments

ftp_sync.py (5.6 KB) - added by schwa 5 years ago.
ftp_sync.py which should fix tickets #62 and #63

Change History

comment:1 follow-up: ↓ 5 Changed 5 years ago by schwa

  • Keywords ftp_sync, sync, Windows added
  • Status changed from new to assigned
  • Milestone set to 2.7

Thanks for the report.

If I understand correctly, you try to sync a file some.file in the directory dir from a Windows client machine to a remote machine. On the remote machine, that file shows up in the directory dir with the name dir\some.file. Is that right?

Can you possibly tell me what's the OS of the remote machine and possibly the FTP server there?

It would be great if you could attach a small runnable script which shows the problem.

comment:5 in reply to: ↑ 1 Changed 5 years ago by ftputiluser

Replying to schwa:

Thanks for the report.

If I understand correctly, you try to sync a file some.file in the directory dir from a Windows client machine to a remote machine. On the remote machine, that file shows up in the directory dir with the name dir\some.file. Is that right?

Can you possibly tell me what's the OS of the remote machine and possibly the FTP server there?

It would be great if you could attach a small runnable script which shows the problem.

Thanks for the report. If I understand correctly, you try to sync a file some.file in the directory dir from a Windows client machine to a remote machine. On the remote machine, that file shows up in the directory dir with the name dir\some.file. Is that right? Can you possibly tell me what's the OS of the remote machine and possibly the FTP server there? It would be great if you could attach a small runnable script which shows the problem.

Thanks for looking into this. Below is the reproduce scenario:

  • Remote FTP Server: Mac OS X Lion
  • Local Client: Window 7
  • ftputil verison: current tip of hg repository
  • Local directory: dir/some.file

After ftp_sync, remote FTP Server's <curr dir> is:

  • <curr dir>/dir/
  • <curr dir>/dir\some.file

that is, a directory under <curr dir> "<curr dir>/dir/" and a file under <curr dir> directly "<curr dir>/dir\some.file"

the "\" is a valid file name char under Mac(*nix system), so ftp server just create the file "dir\some.file" directly under the <curr dir>

The right behavior should be upload "some.file" under "<curr dir>/dir/"

The script is as simple as:

with FTPHost(host, user, password) as remote:
    local = LocalHost()
    syncer = Syncer(local, remote)
    syncer.sync(local_directory, remote_path)

My quick fix is add following line to ftputil/ftp_file.py line 66:

    def _open(self, path, mode):
        """Open the remote file with given path name and mode."""
        # Check mode.
        path = path.replace(os.path.sep, '/') # Fix path error under win32

        if 'a' in mode:

comment:6 Changed 5 years ago by schwa

I don't have a Windows system here, so I can't reproduce your problem. Before I'm guessing at a fix, I'd like to check if what I think happens actually happens. ;-)

Can you please uncomment the print statements in ftputil/ftp_sync.py (in _mkdir and _sync_file), re-run your script, and attach the output to this ticket?

Alternatively, you may put the output in the ticket comment, but in that case really make sure that you wrap the output in triple braces (see WikiFormatting under heading "Preformatted Text"). You can check the display with the "Preview" button before you submit your comment to the ticket system permanently.

comment:7 follow-up: ↓ 8 Changed 5 years ago by ftputiluser

I uncomment the print statement and here's the re-run results:

Making /upload
Making /upload\dir
Syncing e:\a\dir\file.name -> /upload\dir\file.name

One more thing, when upload to filezilla FTP server on windows, the upload path is correct. however, upload to Mac(Unix) FTP, the upload file path is "dir\file.name"

I think to maxmize portability it's better to use portable file seperator "/"

comment:8 in reply to: ↑ 7 Changed 5 years ago by schwa

Would you please replace your version of ftp_sync.py with the file I've attached? Ideally, the attached version should fix both tickets #62 and #63. Can you please put the resulting outputs of your test runs in the respective tickets?

Replying to ftputiluser:

I think to maxmize portability it's better to use portable file seperator "/"

I agree completely. I was only thinking about where to make this change. I had thought about using your fix of replacing the path separator in FTPHost.open, but eventually came to the conclusion that the user of ftputil should be responsible for passing valid arguments to the open method. Of course, ftp_sync is a different story, because a user of the module can only influence the "root" directories for source and target. So, in a sense, ftp_sync here is the FTPHost "user" which has to pass valid arguments to mkdir and open. ;-)

Changed 5 years ago by schwa

ftp_sync.py which should fix tickets #62 and #63

comment:9 Changed 5 years ago by schwa

I revised the file to always change the separators from the source to the target system, not only for uploads to an FTP server.

Note: If you click on the file name link "ftp_sync.py", trac will show the file as an HTML page. To download the actual source file, use the link "Original Format" at the end of that page.

comment:10 follow-up: ↓ 11 Changed 5 years ago by ftputiluser

The attached file fixed #62 but not #63. Here's the output for uploading from win32 to Mac FTP server:

Making /upload
Fixing inner_target_dir from /upload\.hg to /upload/.hg
Making /upload/.hg
Fixing target_file from /upload\.hg\file.name to /upload/.hg/file.name
Syncing e:\a\.hg\file.name -> /upload/.hg/file.name

The file and directory are both correctly created in the their right path.

It might be helpful to document that, the user is responsible for provide the portable path format to use the FTPHost function.

I have to change:

    with FTPHost(host, user, password) as remote:
        with remote.file(os.path.join('a', 'b'), 'wb') as f:
            f.write('1')

to:

    with FTPHost(host, user, password) as remote:
        with remote.file(remote.path.join('a', 'b'), 'wb') as f:
            f.write('1')

to create a correct file from Win32 to Mac FTP

comment:11 in reply to: ↑ 10 Changed 5 years ago by schwa

Replying to ftputiluser:

The attached file fixed #62 but not #63.

Ok, that's something at least. :-) I've committed the fix for #62 as changeset [d70661abd9b7].

It might be helpful to document that, the user is responsible for provide the portable path format to use the FTPHost function.

I've added a paragraph on this to the documentation in changeset [5eb53d367a7b].

I have to change: [...] to create a correct file from Win32 to Mac FTP

Yes, I think that's the approach to take.

comment:12 Changed 5 years ago by schwa

  • Status changed from assigned to closed
  • Resolution set to fixed
Note: See TracTickets for help on using tickets.