How to configure environment for Python Poetry based project.

How do I get started?

I recently switched over to Poetry as a package manager for my project CfnMason. ( CfnMason is a tool related to CloudFormation stack management, but you can read about that on the Readme as it is updated. ) The question is, how do you use Poetry when you are working on a project across multiple machines and operating systems. I guess I am going to attempt to address the issue.

So, you want to join a project, or work on a project that is using the Poetry dependency management tool. Great! But, how do you get the requirements setup for the project so that you can start working on it? How do you know which version of Python to use, which packages to install, how to build the project, or how to run the test suite?

This is an issue that I was facing, but it was not with another project, but my own as I was switching between machines. Now, I do know some of the answers to the questions above, but I was still stuck as to how to setup a project on another machine. As such, I decided to walk through the process of coming onto a new project and determining how to work with it. The project that will be used for this walk-through will be CfMason, at tool that manages some aspects of building and deploying CloudFormation stacks on AWS.

Hopefully the project that you are working on has a Readme file. Though, to be fair, documentation is hard, and is often the last thing that is added to a project. If it does, you should be able them, but if they are not provided, then the following steps are the way that I would go about working on a project that uses Poetry for dependency management. Oh, and as a note. I am making the assumption that you already have Poetry installed.

Steps to work on Poetry based Project

1. Determine that the project is using Poetry
2. Check the version of Python that is needed
   1. Validate local Python version
   2. Install if missing
3. Create a virtual environment for building the application
   1. venv — for packaging and validation
   2. venvdev — for building the package
4. Install project dependencies
5. Build the project
6. Run tests

Determine the project is using Poetry.

If the ReadMe does not tell you that the project is using poetry, then there is a quick way to find out.

• Look for the file pyproject.toml in the base of the project.
• Open the file and look for the following line
  • [tool.poetry]

Provided you are able to find this file, and line, then the project is using Poetry.

Determine the version of Python

One nice thing about Poetry is that it has a defined location to identify the version of Python. I am a big fan of this, as the difference between different versions can cause major problems. Take for example that reserved keywords changed a bunch between 3.6 and 3.7. The steps to follow are as follows.

• Open the toml project file pyproject.toml
• Find the supported python versions in the [tool.poetry.dependencies]
• Find the line starting with python, and find the supported versions.
  • Ex. python = "^3.6"
• If you don’t have that version of Python on your system, install it.

Setting up a local Virtual Env

I am a huge proponent of using a virtual environment for each application that I am working on. In some cases, I will have 2, one for including all the development modules, and one for just the ones needed for the application to run. Since Python 2 is pretty much EOL, I am not going to spend any time on how to setup a virtual environment for Python2. Instead, this is all dedicated to Python3. And I can only guarantee this on Python 3.6 or later.

foo$ python -m venv venv-dev
foo$ python -m venv venv
foo$ ls -ld venv*
drwxr-xr-x 1 foobar 197610 0 Nov 24 17:02 venv/
drwxr-xr-x 1 foobar 197610 0 Dec 25 14:19 venv-dev/

Install project dependencies

For this last part, you need to activate either of the Python Virtual Environments and then run the install code from there. This is only if you really want to install it both ways. If not, then you can just create a single virtual env directory and just install all the dependencies.

Install all the dependencies, even the ones needed for development.

foo$ poetry install
Installing dependencies from lock file


Package operations: 10 installs, 0 updates, 0 removals

  - Installing more-itertools (7.2.0)
  - Installing zipp (0.6.0)
.....
  - Installing cfnmason (0.1.0)

The other option is to just install the libraries needed to execute and run the module. I would almost prefer if it defaulted to the method below, but it works.

foo$ poetry install --no-dev
Installing dependencies from lock file

Nothing to install or update

  - Installing cfnmason (0.1.0)

Start working on the Project

That is it. You should be up and running. At least to the point where you can get started with the project. Moving forward from this point will rely a lot upon how the project is setup, and how well it is documented. But, the big factor is that you can now start working on it while using Poetry, or you have the foundation to work on a project across multiple machines.