Timeline



Feb 17, 2021:

8:34 PM Milestone 5.0.0 completed
8:32 PM Changeset [2062:1a71a77c4f89] by Stefan Schwarzer <sschwarzer@…>
Fix tar.gz directory I guess I made the letter uppercase accidentally.
8:21 PM Changeset [2061:6eb8285406e0] by Stefan Schwarzer <sschwarzer@…>
Use Fedora 33 as image
8:14 PM Changeset [2060:77f3111483f1] by Stefan Schwarzer <sschwarzer@…>
Added tag release_5_0_0 for changeset 9062908e1c7d
8:13 PM ftputil-5.0.0.tar.gz attached to Download by schwa
7:56 PM Changeset [2059:9062908e1c7d] by Stefan Schwarzer <sschwarzer@…>
Add XXX on list announcement and spam
7:53 PM Documentation edited by schwa
(diff)
7:51 PM Download edited by schwa
(diff)
7:43 PM Changeset [2058:84c0806ec734] by Stefan Schwarzer <sschwarzer@…>
Update version and release date
7:42 PM Changeset [2057:a33bbab202fb] by Stefan Schwarzer <sschwarzer@…>
Change version to 5.0.0 (from 5.0.0-alpha)
7:41 PM Changeset [2056:e5ca1e941312] by Stefan Schwarzer <sschwarzer@…>
Remove `-alpha` suffix The alpha version is the new final version. The previous version was only "alpha" in case I needed to change the API again. The code quality should be production level.

Feb 7, 2021:

10:13 PM Changeset [2055:6dacebd7afde] by Stefan Schwarzer <sschwarzer@…>
Increase year in copyright line

Feb 1, 2021:

7:37 PM Changeset [2054:1d80db315313] by Stefan Schwarzer <sschwarzer@…>
Change old version number to 4.0.0
7:19 PM Download edited by schwa
(diff)
7:18 PM Changeset [2053:90572dc48ead] by Stefan Schwarzer <sschwarzer@…>
Adapt release tag format for Semver
7:18 PM Changeset [2052:44fd9795c26f] by Stefan Schwarzer <sschwarzer@…>
Added tag release_5_0_0_alpha for changeset ac94bc654731
7:16 PM Changeset [2051:ac94bc654731] by Stefan Schwarzer <sschwarzer@…>
Remove unused `sys` import
7:14 PM PreReleaseDocumentation edited by schwa
(diff)
7:01 PM Changeset [2050:f2974f53005e] by Stefan Schwarzer <sschwarzer@…>
Patch version info for release
7:00 PM Changeset [2049:b5fb0d2b25c1] by Stefan Schwarzer <sschwarzer@…>
Update version for release 5.0.0-alpha
6:58 PM Changeset [2048:0de064144674] by Stefan Schwarzer <sschwarzer@…>
Update README/announcements for version 5.0.0-alpha The `-alpha` suffix is only there in case I want to change any APIs. The software quality should already be production-level, as usual.
5:57 PM Changeset [2047:0a801e8f639c] by Stefan Schwarzer <sschwarzer@…>
Describe new behavior for missing encoding Raise `NoEncodingError` instead of falling back to latin-1. ticket: 143
5:47 PM Changeset [2046:e59335e8b934] by Stefan Schwarzer <sschwarzer@…>
Raise `NoEncodingError` for absent encoding If an FTP session instance doesn't have an `encoding` attribute, let `FTPHost` raise a `NoEncodingError`. This is a new exception. Before, the code would fall back to the ftputil default encoding, latin-1. This could be surprising behavior for some ftputil users. So rather fail explicitly and early if no `encoding` attribute exists. Otherwise, it the fallback encoding had been wrong, we might get exceptions only much later in production. ticket: 143
5:10 PM Changeset [2045:76ad27c72125] by Stefan Schwarzer <sschwarzer@…>
Sort exceptions in `__all__` alphabetically
3:24 PM Changeset [2044:8fc721629792] by Stefan Schwarzer <sschwarzer@…>
Move paths section after session factory section ticket: 143
3:19 PM Changeset [2043:d268915b0b9f] by Stefan Schwarzer <sschwarzer@…>
Rewrite section of directory/file paths ticket: 143

Jan 28, 2021:

6:05 PM Changeset [2042:ddaa0b9bbaee] by Stefan Schwarzer <sschwarzer@…>
Allow `encoding` argument only for `FTP` subclasses If an explicit encoding, i. e. a `str` is passed to `session_factory`, the `base_class` argument must be `ftplib.FTP` or a subclass of it. The reason is that we already use a heuristic for Python 3.8 and earlier vs. Python 3.9 and earlier. This heuristic may work or fail - possibly silently - for classes that aren't a subclass of `ftplib.FTP` (including `ftplib.FTP` itself). With this check in place, the heuristic may look safe, but it actually isn't if, for example, a class inherits from `ftplib.FTP` in Python 3.9, but overrides the constructor so that it doesn't support the `encoding` argument anymore. ticket: 143
1:10 PM Changeset [2041:66da304cf6ab] by Stefan Schwarzer <sschwarzer@…>
Restore tests under Python 3.9 ticket: 143
12:58 PM Changeset [2040:2f7ddf3d1112] by Stefan Schwarzer <sschwarzer@…>
Add `requirements.txt` file This is just a snapshot from my environment, so different versions may work fine as well.
12:56 PM Changeset [2039:13a2942e3285] by Stefan Schwarzer <sschwarzer@…>
Fix tests for Python 3.9 ticket: 143
12:56 PM Changeset [2038:6022e65454eb] by Stefan Schwarzer <sschwarzer@…>
Add `__all__` ticket: 143
12:28 PM Changeset [2037:25cd7d1417f9] by Stefan Schwarzer <sschwarzer@…>
Fix encoding in comparison The `MockSession` uses the `ftplib` default encoding and the semantics of `session_factory` is to use the default encoding of the base class if no `encoding` argument is specified for `session_factory`. ticket: 143
12:25 PM Changeset [2036:5a655d4cf373] by Stefan Schwarzer <sschwarzer@…>
Fix `test_path.py` for Python 3.9 Use the same approach as in the fix for `test_host.py` (see previous commit). ticket: 143
12:14 PM Changeset [2035:d1192e577ef2] by Stefan Schwarzer <sschwarzer@…>
Fix `test_host.py` for Python 3.9 Add an optional argument `encoding` argument to the `as_bytes` helper function and use this argument in `TestAcceptEitherUnicodeOrBytes`. The code in `scripted_session` uses the path encoding of the `ftplib` module in the standard library, whereas the `as_bytes` helper function in `test_host.py` by default uses the ftputil path encoding default. Therefore, the tests worked for Python 3.8 and before, but not for Python 3.9 with the changed path encoding default. ticket: 143
11:52 AM Changeset [2034:4b81570af0f3] by Stefan Schwarzer <sschwarzer@…>
Use new module `path_encoding` Replace hardcoded uses of "latin-1", "utf-8" and Python version checks with constants from `ftputil.path_encoding`. ticket: 143

Jan 27, 2021:

7:22 PM Changeset [2033:99fb8a5f1ab6] by Stefan Schwarzer <sschwarzer@…>
Make path encoding configurable Some changes: - Add `default_session_factory` in `host.py` and use the new session factory as the default factory instead of `ftplib.FTP`. This makes sure that we keep the old behavior of ftputil under Python 3.8 and below, that is, using latin-1 path encoding. - When creating an instance from the session factory, store the encoding of the instance in the `FTPHost` instance in the `_encoding` attribute. In case the created session has no `encoding` attribute, fall back to the default, i. e. "latin-1". - Add an `encoding` argument to the string conversion tools in `tool.py`. - When calling into `tool.py`, pass the `_encoding` from the `FTPHost` instance as the `encoding` argument. There are a few places that call into `tool.py` without the session encoding being available. This applies to `error.py`. In this case use the default encoding, latin-1. - Adapt tests. For example, make sure that some mock FTP sessions have an `encoding` attribute set. With these changes, the tests run successfully under Python 3.6 to 3.8. There are still a few tests failing under Python 3.9. ticket: 143
5:50 PM Changeset [2032:d36882f78aae] by Stefan Schwarzer <sschwarzer@…>
Add encoding attribute Otherwise, the `encoding` access goes through `__getattr__`. We want to avoid this because we would have to add `encoding` accesses to all session scripts. Let's treat `encoding` as a regular attribute instead, without tracking it with the session logic for expected attribute accesses. ticket: 143
5:49 PM Changeset [2031:aa15029facb4] by Stefan Schwarzer <sschwarzer@…>
Print next `Call` in scripted session
5:49 PM Changeset [2030:3fd92baa7cb0] by Stefan Schwarzer <sschwarzer@…>
Remove brackets from some scripted session output
4:29 PM Changeset [2029:6140fb27fe6e] by Stefan Schwarzer <sschwarzer@…>
Use base class default if `encoding` isn't given Make the `encoding` argument of `session_factory` default to `None`. If `None`, use the default encoding of the base class. This follows the same logic as for the default handling of the `use_passive_mode` argument. Note that the default encoding of `ftplib.FTP` changed from "latin-1" in Python <= 3.8 to "utf-8" in Python >= 3.9. Therefore, if you want the same encoding regardless of the Python version, pass your own base class (compatible with `ftplib.FTP`) which uses a fixed encoding or pass the `encoding` explicitly to `session_factory`. ticket: 143
3:47 PM Changeset [2028:72a34b226c8d] by Stefan Schwarzer <sschwarzer@…>
Add `encoding` argument for `session_factory` ticket: 143

Jan 6, 2021:

10:33 PM Changeset [2027:b0f43547abc7] by Stefan Schwarzer <sschwarzer@…>
Make new image the version 0.3 (see recent image changes)
8:36 PM Changeset [2026:806db56bceaf] by Stefan Schwarzer <sschwarzer@…>
Update README (port mapping)
6:33 PM Changeset [2025:d9aae7324166] by Stefan Schwarzer <sschwarzer@…>
Don't run tests for Python 3.9 The integration tests currently fail for Python 3.9 because of the changed default path encoding (see ticket #143).
5:54 PM Changeset [2024:5b5ac0e7f191] by Stefan Schwarzer <sschwarzer@…>
Run in privileged mode for Podman to work
5:18 PM Changeset [2023:bd164503fe57] by Stefan Schwarzer <sschwarzer@…>
Rework test server container - Switch to configuration via configuration file. As I understand the Pure-FTPd documentation, configuration happens either by command line switches or by specifying just the path of a configuration file. Before, I didn't notice that the configuration file wasn't used because it seems the command line contained all the necessary switches for my server configuration. Curiously, when using a configuration file, the FTP server no longer runs in the foreground, but is moved to the background. This leads the container to exit as soon as the server is started. To work around this, add `&& sleep infinity` to the `CMD`. The `infinity` argument may not be supported in all versions of `sleep`, but for the Debian base image of the Pure FTPd image, the argument _is_ supported. A obscure workaround would be `&& tail -f /dev/null`. - Let the server in the container listen on port 2121 instead of port 21. This should avoid network capability problems when starting the server. Strangely, I hadn't noticed this problem before. Since it happens now (for port 21) with both Docker and Podman on my machine, the trigger may be in a common component, say, runc.
5:16 PM Changeset [2022:d8fc41786c69] by Stefan Schwarzer <sschwarzer@…>
Don't use color output when running under tox In the Sourcehut build system, Pytest color output leads to a messy log with ANSI color codes.
5:11 PM Changeset [2021:2b9889bc49d0] by Stefan Schwarzer <sschwarzer@…>
Prepend image name with `localhost/` This is supposed to avoid registry lookups for an actually local image. I saw such lookups in the Sourcehut build system.

Jan 5, 2021:

7:33 PM Changeset [2020:cc5bcc816605] by Stefan Schwarzer <sschwarzer@…>
Comment out tests under Python 3.9

Dec 8, 2020:

1:25 PM Ticket #144 (Add `FTPHost.copytree`) created by schwa
Add FTPHost.copytree, similar to shutil.copytree. See …

Dec 1, 2020:

6:06 PM Changeset [2019:016c9d1d0e8f] by Stefan Schwarzer <sschwarzer@…>
Test Python 3.6 to 3.9 with tox
2:25 PM Ticket #143 (Encoding-related failure with Python 3.9) created by schwa
When running the tests with tox for Python 3.9, I get […] I assume …
1:59 PM Changeset [2018:59b78101f450] by Stefan Schwarzer <sschwarzer@…>
Test on Python 3.9

Nov 30, 2020:

10:20 PM Ticket #142 (Make container for integration tests) closed by schwa
fixed: I revised the Dockerfile to not use ADD so that it works with …
9:23 PM Changeset [2017:ab68a8416f5f] by Stefan Schwarzer <sschwarzer@…>
Add build manifest for Sourcehut
9:15 PM Changeset [2016:1eab96c8af79] by Stefan Schwarzer <sschwarzer@…>
Add `.empty` marker
8:58 PM Changeset [2015:cf2177ee054b] by Stefan Schwarzer <sschwarzer@…>
Add `.empty` marker
8:42 PM Changeset [2014:4d943001bbdd] by Stefan Schwarzer <sschwarzer@…>
Add `.empty` file Add `.empty` file to retain directory in Mercurial repository.
7:49 PM Changeset [2013:c2712bb002de] by Stefan Schwarzer <sschwarzer@…>
Add `-a` option to `container ls`
7:45 PM Changeset [2012:ffb473bc8263] by Stefan Schwarzer <sschwarzer@…>
Change image tag to 0.2 This should help users a bit not using an obsolete image.
7:42 PM Changeset [2011:4be14c0d42a7] by Stefan Schwarzer <sschwarzer@…>
Add (and remove) `.empty` files These files are needed so that the directory structure is recreated when cloning/updating the Mercurial repository. On the other hand, we don't actually want the `.empty` files in the container, so remove them in the `RUN` command.
7:18 PM Changeset [2010:fa4b91eec358] by Stefan Schwarzer <sschwarzer@…>
Comment out `ftp.gnome.org` This currently (or permanently?) fails with "Network unreachable".
7:07 PM Changeset [2009:6d56d76e59ab] by Stefan Schwarzer <sschwarzer@…>
Remove unused files `rest_test.txt` is _created_ in one of the tests. `sync_test` doesn't seem to be used at all. Update `CONTENTS` (although the actual content doesn't matter for the tests).
6:58 PM Changeset [2008:8ae153555894] by Stefan Schwarzer <sschwarzer@…>
Rename `debian-keyring.tar.gz` to `large_file` This file _used to be_ an actual Debian keyring file, but I truncated it to make the image smaller. We don't need the whole actual file content.
6:50 PM Changeset [2007:c49235b09a67] by Stefan Schwarzer <sschwarzer@…>
Run Black on `find_problematic_code.py`
6:49 PM Changeset [2006:1c1f2b371ab5] by Stefan Schwarzer <sschwarzer@…>
Set image version to 0.2
6:23 PM Changeset [2005:1cf1fe78dd81] by Stefan Schwarzer <sschwarzer@…>
Make set-up of `ftptest` home folder explicit The previous implementation used an `ftptest.tar.gz` file that was a copy of my former local test directory. This approach had several downsides: - To create the `tar.gz` file so that it can be unpacked without modification with `ADD` in the Dockerfile, the owners/groups of the archive contents had to be correct, which required owner map and group map argument for `tar`. - Due to the versioned artifact being the `tar.gz` file and creating it was difficult (see before), it would have been very hard for contributors to make changes to the archive (for example when tests are added or modified). - The use of `ADD` on a file with heterogenous owners/groups triggered a bug when the image was created with Podman (see https://github.com/containers/podman/issues/8064 ). The new approach makes these changes: - The versioned data is the original `ftptest` home directory, so it's relatively easy to modify for new/changed tests. - The `ftptest` directory is copied with `COPY`. - The Dockerfile sets almost all owners/groups to `root:root`. Only where needed owner and group are set to `ftptest:ftpgroup`. This makes it explicit which tests need to modify which paths. - The restricted owner/group settings also have the advantage that tests which shouldn't modify the directory but accidentally did, would obviously fail.

Nov 29, 2020:

8:25 PM Ticket #142 (Make container for integration tests) reopened by schwa
The implementation works, but has problems I'd like to solve: - When …
12:03 AM Changeset [2004:c219752b5d4e] by Stefan Schwarzer <sschwarzer@…>
Set container-related variables conditionally

Nov 28, 2020:

6:04 PM Ticket #142 (Make container for integration tests) closed by schwa
fixed: I have implemented this, see browser:/test_server . You can build the …
5:55 PM Ticket #142 (Make container for integration tests) created by schwa
For years, the tests in test/test_real_ftp.py have run against an …
9:19 AM Changeset [2003:4910616f288d] by Stefan Schwarzer <sschwarzer@…>
Limit container to localhost Otherwise, hosts outside the host running the container can connect to your container!

Nov 27, 2020:

10:56 PM Changeset [2002:1a1ab0ff1745] by Stefan Schwarzer <sschwarzer@…>
Add `.dockerignore` file
10:45 PM Changeset [2001:7b878e67318a] by Stefan Schwarzer <sschwarzer@…>
Add README on image and container
10:30 PM Changeset [2000:d6af4566e0fa] by Stefan Schwarzer <sschwarzer@…>
Update to use with FTP server container See the subdirectory `test_server`.
10:27 PM Changeset [1999:df5ee67aa60d] by Stefan Schwarzer <sschwarzer@…>
Omit `-it` options These aren't necessary since we detach the container anyway.
10:19 PM Changeset [1998:055bbbbba62a] by Stefan Schwarzer <sschwarzer@…>
Unpack test data for integration tests The `ftptest` directory in `ftptest.tar.gz` is the data used by the integration tests. There may be unnessary data that's no longer used by the tests. I also experimented with `COPY`ing the `ftptest` subdirectory directly, but this had problems: - The owner for all files in the destination was `root`. However, it's intentional that some of the files are owned by `ftptest:ftpgroup` and some by `root:root`. - Directory and file names with non-ASCII characters in them were messed up. - There were problems with directory and file names with spaces in them. We need these for testing.
10:18 PM Changeset [1997:c5721b43ab1f] by Stefan Schwarzer <sschwarzer@…>
Add test data for integration tests This file needs to be unpacked in the container image (via the Dockerfile).
10:05 PM Changeset [1996:69a6fb599409] by Stefan Schwarzer <sschwarzer@…>
Execute all tests before a release I usually run `tox` before releases. In this cases it makes sense to run all tests, even if they're slow.
10:03 PM Changeset [1995:f17b8e8afed8] by Stefan Schwarzer <sschwarzer@…>
Detach container for FTP test server
9:59 PM Changeset [1994:7016db098192] by Stefan Schwarzer <sschwarzer@…>
Be more explicit in comment
9:58 PM Changeset [1993:24b7014a4fb8] by Stefan Schwarzer <sschwarzer@…>
Allow CHMOD command (presumably) For some reason, this doesn't work, though, and I need the `-A` option on the command line.
9:48 PM Changeset [1992:b6fe02ff004d] by Stefan Schwarzer <sschwarzer@…>
Chroot anyone but root Somehow the option `NoChroot no` in the configuration file doesn't work as I expected. The command line option works, though.
9:35 PM Changeset [1991:1dd7278644dd] by Stefan Schwarzer <sschwarzer@…>
Allow CHMOD command
8:15 PM Changeset [1990:672e324b15d3] by Stefan Schwarzer <sschwarzer@…>
Add targets to create docker image and container
8:13 PM Changeset [1989:b5dc24f8d863] by Stefan Schwarzer <sschwarzer@…>
Add `Dockerfile` for test server With the current setup, the `ftptest` user can log in and upload a file to their login directory. The test data for the ftputil integration tests isn't included yet.
8:12 PM Changeset [1988:29b2bae665ed] by Stefan Schwarzer <sschwarzer@…>
Use unix users instead of virtual users
4:35 PM Changeset [1987:670e6173429e] by Stefan Schwarzer <sschwarzer@…>
Put `.PHONY` directives above target definitions Collecting the `.PHONY` targets in one place makes the file shorter, but it's too easy to get the targets in the `.PHONY` directive out of sync with the present actual target definitions.
4:20 PM Changeset [1986:9bc4725b15eb] by Stefan Schwarzer <sschwarzer@…>
Add merged `pure-ftpd.conf` This is a combination of `stilliard`s original file and the file I used so far for my local FTP server.
Note: See TracTimeline for information about the timeline view.