When trying to open a file on a mvs-host for writing with the following commands
>>> from ftputil import *
>>> datei = FTPHost("hostname", "username", "password").open("versuch", "w")
I get the following traceback:
Traceback (most recent call last):
File "<pyshell#1>", line 1, in <module>
datei = FTPHost("hostname", "username", "password").open("versuch", "w")
File "C:\Program Files\Python25\lib\site-packages\ftputil\ftputil.py", line 230, in open
return self.file(path, mode)
File "C:\Program Files\Python25\lib\site-packages\ftputil\ftputil.py", line 225, in file
self.stat_cache.invalidate(effective_path)
File "C:\Program Files\Python25\Lib\site-packages\ftputil\ftp_stat_cache.py", line 128, in invalidate
assert path.startswith("/"), "%s must be an absolute path" % path
AssertionError: 'username.'/versuch must be an absolute path
There is no problem to open a (existing) file with the above commandline for reading. The problem seems to be the naming convention on mvs systems where no root directory called / exists. There ist an ugly workaround for this problem:
datei = FTPHost("hostname", "username", "password").open("/'username.versuch'", "w")
forces the / to be the first character of the filename which is required by ftp_stat_cache.py.
Thanks for your issue report.
Could you please substitute the method FTPHost.file with the following debug version and run the code in your bug description (above) again? Please add a comment containing the output.
def file(self, path, mode='r'): """ Return an open file(-like) object which is associated with this `FTPHost` object. This method tries to reuse a child but will generate a new one if none is available. """ host = self._available_child() if host is None: host = self._copy() self._children.append(host) host._file = ftp_file._FTPFile(host) basedir = self.getcwd() print "basedir:", basedir # prepare for changing the directory (see whitespace workaround # in method `_dir`) if host.path.isabs(path): print "is absolute" effective_path = path else: print "is not absolute" effective_path = host.path.join(basedir, path) print "effective path:", effective_path effective_dir, effective_file = host.path.split(effective_path) print "effective dir:", effective_dir print "effective file:", effective_file try: # this will fail if we can't access the directory at all host.chdir(effective_dir) except ftp_error.PermanentError: # similarly to a failed `file` in a local filesystem, we # raise an `IOError`, not an `OSError` raise ftp_error.FTPIOError("remote directory '%s' doesn't exist " "or has insufficient access rights" % effective_dir) host._file._open(effective_file, mode) if 'w' in mode: self.stat_cache.invalidate(effective_path) return host._file
Stefan
Hi Stefan, here is the result (traceback omitted):
FTPHost("hostname", "username", "password").file("versuch", "w") basedir: 'username.' is not absolute effective path: 'username.'/versuch effective dir: 'username.' effective file: versuch
Regards, Axel
Hi Axel,
In comparison to Posix or Windows, this path syntax is quite unfamiliar to me. I don't have an idea yet to support this out-of-the-box and cleanly in ftputil.
Some questions that might help:
- Can you configure the FTP server? In particular, can you configure the server to emit Posix path names instead of the MVS syntax?
- Is the path syntax used by the server the same as that of MVS' file system?
- Are there resources about the path syntax on the web?
- Can you tell something about the FTP server software? What's its name and version?
- What is the output of the FTP STAT, SITE and SYST [chttp://www.nsftools.com/tips/RawFTP.htm commands] if the server accepts them? You can invoke the commands via ftplib 's sendcmd command.
As you set the priority of this bug report to "minor" it might suffice to apply a workaround without modifying ftputil. For example, could you write a utility function to change the path as you described in the initial report?
Stefan
Hi Stefan, here are the answers to your questions:
1st I can't configure the FTP server, I am just an ordinary user on the host.
2nd/3rd As far as I understand MVS the is nothing like a path structure at all. I found a short description in the german wikipedia:
Hier ist Dataset eine andere Bezeichnung für eine Datei, die auf einem IBM-Großrechnersystem existiert. Ein Dataset-Name (DSN) kann maximal 44 Zeichen lang sein und besteht aus mehreren Qualifiern (Namensteilen), die durch Punkte voneinander getrennt werden. Jeder Qualifier kann maximal acht Stellen lang sein. Beispiel: MEIN.PRIVATES.TEST.DATASET.V1
http://de.wikipedia.org/wiki/Dataset
If the DSN is not surrounded by apostrophes (') the name of the current user is added as the first qualifier (i.e. MEIN.DATASET.V1 is the same as 'USERNAME.MEIN.DATASET.V1' ). Additionally, the DSNs can include a member name in braces: 'USERNAME.DATASET.V1(MEMBER1)' )
The FTP server has a meaning of a current directory which is described by adding the current "path", which consists of one or more qualifiers, to a given name.
4th/5th
>>> print xxx.sendcmd("stat") 211-Server FTP talking to host xx.yy.zz.aa, port 1858 211-User: xyz Working directory: xyz. 211-The control connection has transferred 2636 bytes 211-There is no current data connection. 211-The next data connection will be actively opened 211-to host xxx.yy.zz.aa, port 1858, 211-using Mode Stream, Structure File, type ASCII, byte-size 8 211-Automatic recall of migrated data sets. 211-Automatic mount of direct access volumes. 211-Auto tape mount is allowed. 211-Inactivity timer is set to 700 211-Timer FTPKEEPALIVE is set to 3600 211-VCOUNT is 59 211-ASA control characters in ASA files opened for text processing 211-will be transferred as ASA control characters. 211-Trailing blanks are not removed from a fixed format 211-data set when it is retrieved. 211-Data set mode. (Do not treat each qualifier as a directory.) 211-ISPFSTATS is set to FALSE 211-Primary allocation 5 cylinders. Secondary allocation 5 cylinders. 211-Partitioned data sets will be created with 15 directory blocks. 211-FileType SEQ (Sequential - default). 211-Number of access method buffers is 5 211-RDWs from variable format data sets are discarded. 211-Records on input tape are unspecified format 211-SITE DB2 subsystem name is DB2 211-Data not wrapped into next record. 211-Tape write is not allowed to use BSAM I/O 211-Truncated records will not be treated as an error 211-JESLRECL is 80 211-JESRECFM is Fixed 211-JESINTERFACELEVEL is 1 211-Server site variable JESTRAILINGBLANKS is set to TRUE 211-Confidence level in data transfers is neither checked nor reported 211-ENcoding is set to SBCS 211-Outbound SBCS ASCII data uses CRLF line terminator 211-Outbound MBCS ASCII data uses CRLF line terminator 211-Server site variable MBREQUIRELASTEOL is set to TRUE 211-Server site variable UNICODEFILESYSTEMBOM is set to ASIS 211-DBSUB is set to FALSE 211-SBSUB is set to FALSE 211-SBSUBCHAR is set to SPACE 211-SMS is active. 211-Mgmtclass for new data sets is MCS30570 211-Data sets will be allocated using unit SYSDA 211-New data sets will be catalogued if a store operation ends abnormally 211-Single quotes will override the current working directory. 211-UMASK value is 027 211-Process id is 50791100 211-Checkpoint interval is 0 211-Authentication type: None 211-Port of Entry resource class for IPv4 clients is: TERMINAL 211-Record format FB, Lrecl: 80, Blocksize: 29440 211-Server site variable LISTSUBDIR is set to TRUE 211 *** end of status ***
>>> print xxx.sendcmd("site") 202 SITE not necessary; you may proceed
>>> print xxx.sendcmd("syst") 215 MVS is the operating system of this server. FTP Server is running on z/OS.
Of course it would be possible to write a helper function to mask the special mvs behaviour. But of course, I would prefer a solution in the ftplib itself. If you need more information or debugging support, don't hesitate to ask me.
Best regards, Axel
Hello Axel,
sorry, I forgot to follow up.
I think I don't want to add a larger additional abstraction level to ftputil's navigation. However, if you can think of a way to do add the MVS compatibility with minimal drawbacks to the code's clarity/understandability, you may suggest possible approaches.
Stefan