📦 Create Custom Python Package¶

Packing allows us to modularize the code and reused them access the projects. Additionally, it allows us to test them independently. Refer to Getting Started With Testing in Python and Effective Python Testing With Pytest.

The general package structure would look like:

Text Only

project/
│
├── src/
│   └── auth/
│   |   ├── __init__.py
│   |   ├── __main__.py
│   |   ├── config.toml
│   |   ├── feed.py
│   |   └── viewer.py
|   |__ utils/
|       |__ __init__.py    
│
├── tests/
│   ├── test_auth.py
│   └── test_utils.py
│
├── LICENSE
├── MANIFEST.in
├── README.md
└── pyproject.toml

Let's create a package just containing two files inside the app/src folder.

Requirements¶

Python installed

Init src folder as module with init.py folder in it¶

Create setup.py file¶

setup.py

from setuptools import setup, find_packages

setup(
    #this will be the package name you will see, e.g. the output of 'conda list' in anaconda prompt
    name = 'custom_package', 
    #some version number you may wish to add - increment this after every update
    version='1.0', 

    # Use one of the below approach to define package and/or module names:

    #if there are only handful of modules placed in root directory, and no packages/directories exist then can use below syntax
#     packages=[''], #have to import modules directly in code after installing this wheel, like import mod2 (respective file name in this case is mod2.py) - no direct use of distribution name while importing

    #can list down each package names - no need to keep __init__.py under packages / directories
#     packages=['<list of name of packages>'], #importing is like: from package1 import mod2, or import package1.mod2 as m2

    #this approach automatically finds out all directories (packages) - those must contain a file named __init__.py (can be empty)
    packages=find_packages(), #include/exclude arguments take * as wildcard, . for any sub-package names
    install_requires=[
        "schedule"
    ]
)

Install build tools¶

PowerShell

python/py -m pip install build twine

Build the package¶

Text Only

py -m build

It would generate .whl and tar.gz file Successfully built custom_package-1.0.tar.gz and custom_package-1.0-py3-none-any.whl

Use the package in Synapse workspace¶

The package can be uploaded in Azure synapse workspace. Once uploaded that package can be installed in the selected pool. Therefore, package has to be exclusively added in pool packages.

Reference: Create and publish python package

References¶

Parsing python config