Lecture 05
In Python, a module is simply a file containing Python code (.py extension).
Modules provide:
The import statement loads a module (i.e. runs the code) and creates a namespace for its contents,
There are several additional ways to import modules or their contents:
The from module import * pattern imports all public names (does not start with _) but is considered bad practice as it can lead to namespace pollution and naming conflicts.
When you import a module, Python searches for it in the following order:
PYTHONPATH environment variableYou can inspect the search path via sys.path,
[ /Users/rundel/.local/share/uv/python/cpython-3.14.2-macos-aarch64-none/bin,
/Users/rundel/.local/share/uv/python/cpython-3.14.2-macos-aarch64-none/lib/python314.zip,
/Users/rundel/.local/share/uv/python/cpython-3.14.2-macos-aarch64-none/lib/python3.14,
/Users/rundel/.local/share/uv/python/cpython-3.14.2-macos-aarch64-none/lib/python3.14/lib-dynload,
/Users/rundel/Desktop/Sta663-Sp26/website/.venv/lib/python3.14/site-packages,
/Users/rundel/Library/R/arm64/4.5/library/reticulate/python,
/Users/rundel/Desktop/Sta663-Sp26/website/.venv/lib/python314.zip,
/Users/rundel/Desktop/Sta663-Sp26/website/.venv/lib/python3.14,
/Users/rundel/Desktop/Sta663-Sp26/website/.venv/lib/python3.14/lib-dynload ]
Modules have several built-in attributes that provide useful metadata,
'json'
'/Users/rundel/.local/share/uv/python/cpython-3.14.2-macos-aarch64-none/lib/python3.14/json/__init__.py'
JSON (JavaScript Object Notation) <https://json.org> is a subset of
JavaScript syntax (ECMA-262 3rd edition) used as a lightweight data
interchange format.
:mod:`json` exposes an API familiar to user ...
dir() functionThe dir() function lists all names defined in a module (or any object),
['__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', 'greet', 'pi']
['JSONDecodeError', 'JSONDecoder', 'JSONEncoder', '__all__', '__author__', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__', '__version__', '_default_decoder', '_default_encoder', 'codecs', 'decoder', 'detect_encoding', 'dump', 'dumps', 'encoder', 'load', 'loads', 'scanner']
__name__ and "__main__"Every module has a __name__ attribute. When a module is run directly, __name__ is set to "__main__". When imported, it’s set to the module’s name.
Lec05_app.py
A package is a way to organize related modules into a hierarchal structure using directories. A package is a directory containing:
__init__.py file (can be empty).py)__init__.py).
└── data_tools
├── __init__.py
├── io
│ ├── __init__.py
│ ├── csv_reader.py
│ └── json_reader.py
├── analysis
│ ├── __init__.py
│ ├── statistics.py
│ └── visualization.py
└── utils.py
__init__.pyThe __init__.py file:
__all__ to control from package import *__init__.py examplefrom .__version__ import __version__
from ._api import delete, get, head, options, patch, post, put, request, stream
from ._auth import BasicAuth, DigestAuth, NetRCAuth
from ._client import AsyncClient, Client
from ._config import Limits, Proxy, Timeout, create_ssl_context
from ._exceptions import (ConnectError, ConnectTimeout, HTTPError, HTTPStatusError, ...)
from ._models import Cookies, Headers, Request, Response
from ._transports import AsyncHTTPTransport, HTTPTransport, MockTransport
from ._urls import URL, QueryParams
__all__ = [
"__version__",
"AsyncClient", "Client",
"delete", "get", "head", "options", "patch", "post", "put", "request",
"BasicAuth", "DigestAuth", "NetRCAuth",
"Limits", "Proxy", "Timeout",
"HTTPError", "ConnectError", "TimeoutException",
"Request", "Response", "Headers", "Cookies", "URL",
...
]Packages allow for hierarchical imports,
ParseResult(scheme='https', netloc='duke.edu', path='/path', params='', query='x=1', fragment='')
PosixPath('/Users/rundel/Desktop/Sta663-Sp26/website/static/slides')
Python comes with a large collection of built-in packages called the standard library - these are available without installing anything extra.
Some useful examples:
json - JSON encoding/decodingpathlib - filesystem pathsos - operating system interfacesys - system-specific parametersre - regular expressionsdatetime - dates and timescollections - specialized containersitertools - iteration utilitiesfunctools - higher-order functionsmath - mathematical functionsrandom - random number generationtyping - type hintsGiven the following package structure (available in exercises/):
.
└── analytics
├── __init__.py
├── data
│ ├── __init__.py
│ ├── cleaner.py # contains: remove_nulls(), normalize()
│ └── loader.py # contains: load_csv(), load_json()
└── models
├── __init__.py
└── regression.py # contains: LinearModel class
Write import statements to:
load_csv from loader.pycleaner.pyLinearModel with the alias lmregression.py, import remove_nulls using a relative importPython’s packaging ecosystem has historically been fragmented:
pip, virtualenv, venv, conda, poetry, pipenv, etc.requirements.txt, setup.py, pyproject.toml, etc.pyenv)uv is a modern tool that aims to unify these concerns with a fast, Rust-based implementation.
uv is a Python package and project manager developed by Astral (creators of ruff).
Key features:
pyproject.tomlpip and virtualenvuv is already installed on the departmental servers, for local installs:
On MacOS/Linux:
or with homebrew:
or with pip / pipx
Once installed you should be able to run the following,
As long as you have version 0.9.* you should be fine.
uv can install and manage multiple Python versions,
cpython-3.15.0a5-macos-aarch64-none <download available>
cpython-3.15.0a5+freethreaded-macos-aarch64-none <download available>
cpython-3.14.2-macos-aarch64-none /opt/homebrew/bin/python3.14 -> ../Cellar/python@3.14/3.14.2/bin/python3.14
cpython-3.14.2-macos-aarch64-none /opt/homebrew/bin/python3 -> ../Cellar/python@3.14/3.14.2/bin/python3
cpython-3.14.2-macos-aarch64-none /Users/rundel/.local/bin/python3.14 -> /Users/rundel/.local/share/uv/python/cpython-3.14.2-macos-aarch64-none/bin/python3.14
cpython-3.14.2-macos-aarch64-none /Users/rundel/.local/share/uv/python/cpython-3.14.2-macos-aarch64-none/bin/python3.14
cpython-3.14.2+freethreaded-macos-aarch64-none <download available>
cpython-3.13.11-macos-aarch64-none /opt/homebrew/bin/python3.13 -> ../Cellar/python@3.13/3.13.11/bin/python3.13
cpython-3.13.11-macos-aarch64-none <download available>
cpython-3.13.11+freethreaded-macos-aarch64-none <download available>
cpython-3.12.12-macos-aarch64-none /opt/homebrew/bin/python3.12 -> ../Cellar/python@3.12/3.12.12/bin/python3.12
cpython-3.12.12-macos-aarch64-none <download available>
cpython-3.11.14-macos-aarch64-none <download available>
cpython-3.10.19-macos-aarch64-none <download available>
cpython-3.9.25-macos-aarch64-none <download available>
cpython-3.9.6-macos-aarch64-none /usr/bin/python3
cpython-3.8.20-macos-aarch64-none <download available>
pypy-3.11.13-macos-aarch64-none <download available>
pypy-3.10.16-macos-aarch64-none <download available>
pypy-3.9.19-macos-aarch64-none <download available>
pypy-3.8.16-macos-aarch64-none <download available>
graalpy-3.12.0-macos-aarch64-none <download available>
graalpy-3.11.0-macos-aarch64-none <download available>
graalpy-3.10.0-macos-aarch64-none <download available>
graalpy-3.8.5-macos-aarch64-none <download available>
The pinned version is stored in ~/.python-version and will be used automatically.
Use uv init to create a new project,
Initialized project `my-project`
total 32
drwxr-xr-x@ 8 rundel staff 256 Jan 21 13:22 .
drwxr-xr-x@ 135 rundel staff 4320 Jan 21 13:23 ..
drwxr-xr-x@ 9 rundel staff 288 Jan 21 13:22 .git
-rw-r--r--@ 1 rundel staff 109 Jan 21 13:22 .gitignore
-rw-r--r--@ 1 rundel staff 5 Jan 21 13:22 .python-version
-rw-r--r--@ 1 rundel staff 82 Jan 21 13:22 main.py
-rw-r--r--@ 1 rundel staff 150 Jan 21 13:22 pyproject.toml
-rw-r--r--@ 1 rundel staff 0 Jan 21 13:22 README.md
This creates a pyproject.toml, a sample main.py script, and basic git infrastructure. Generally, we only really care about the pyproject.toml which we can exclusively generate via uv init --bare.
pyproject.tomlModern project metadata file, tracks python version and package dependencies among other details.
Once we have our project setup we can add (and install) dependencies directly via uv. uv add updates pyproject.toml and installs the package (creating a venv if needed).
Using CPython 3.14.2
Creating virtual environment at: .venv
Resolved 2 packages in 157ms
Installed 1 package in 27ms
+ numpy==2.4.1
Resolved 18 packages in 490ms
Prepared 5 packages in 9.18s
Installed 15 packages in 134ms
+ contourpy==1.3.3
+ cycler==0.12.1
+ fonttools==4.61.1
+ joblib==1.5.3
+ kiwisolver==1.4.9
+ matplotlib==3.10.8
+ packaging==25.0
+ pandas==3.0.0
+ pillow==12.1.0
+ pyparsing==3.3.2
+ python-dateutil==2.9.0.post0
+ scikit-learn==1.8.0
+ scipy==1.17.0
+ six==1.17.0
+ threadpoolctl==3.6.0
Resolved 26 packages in 336ms
Prepared 1 package in 238ms
Installed 2 packages in 3ms
+ pydantic==1.10.26
+ typing-extensions==4.15.0
Resolved 24 packages in 337ms
Prepared 4 packages in 859ms
Installed 5 packages in 27ms
+ iniconfig==2.3.0
+ pluggy==1.6.0
+ pygments==2.19.2
+ pytest==9.0.2
+ ruff==0.14.13
pyproject.toml[project]
name = "my-project"
version = "0.1.0"
description = "Add your description here"
readme = "README.md"
requires-python = ">=3.14"
dependencies = [
"matplotlib>=3.10.8",
"numpy>=2.4.1",
"pandas>=3.0.0",
"pydantic<2",
"scikit-learn>=1.8.0",
]
[dependency-groups]
dev = [
"pytest>=9.0.2",
"ruff>=0.14.13",
]Virtual environments isolate project dependencies from the system Python and other projects. Packages are installed in a local folder in your project.
As we just saw, using uv add will create a new virtual environment in .venv by default if there is not an existing venv.
To explicitly create your own venv you can use,
To use the virtual environment certain environmental variables need to be set correctly (e.g. PATH, PYTHONPATH, etc.) so that the correct Python binary and libraries are used.
From the command line / terminal you can run the following in your project directory:
Alternatively (strongly suggested), use uv run to execute commands in the environment without activating,
Positron automatically detects virtual environments in your project directory. When you open a folder containing a .venv directory (created by uv), Positron will:
If not automatically detected, you can manually select the interpreter via the Command Palette (Cmd+Shift+P / Ctrl+Shift+P) and searching for “Python: Select Interpreter”.
uv syncSince the .venv folder is system specific (and large) it is not typically committed to git. Instead you will likely clone a repository that just has a pyproject.toml file.
Use uv sync to construct the venv and install all dependencies for the project
Using CPython 3.14.2
Creating virtual environment at: .venv
Resolved 26 packages in 8ms
Installed 23 packages in 96ms
+ contourpy==1.3.3
+ cycler==0.12.1
+ fonttools==4.61.1
+ iniconfig==2.3.0
+ joblib==1.5.3
+ kiwisolver==1.4.9
+ matplotlib==3.10.8
+ numpy==2.4.1
+ packaging==25.0
+ pandas==3.0.0
+ pillow==12.1.0
+ pluggy==1.6.0
...
Package Version
----------------- -----------
contourpy 1.3.3
cycler 0.12.1
fonttools 4.61.1
...
Name: numpy
Version: 2.4.1
Location: /Users/rundel/my-project/.venv/lib/python3.14/site-packages
Requires:
Required-by: contourpy, matplotlib, pandas, scikit-learn, scipy
uv pip install and uv pip freeze are also available as lower-level commands for working outside of a project context or with requirements.txt files.
Resolved 26 packages in 8ms
my-project v0.1.0
├── matplotlib v3.10.8
│ ├── contourpy v1.3.3
│ │ └── numpy v2.4.1
│ ├── cycler v0.12.1
│ ├── fonttools v4.61.1
│ ├── kiwisolver v1.4.9
│ ├── numpy v2.4.1
│ ├── packaging v25.0
│ ├── pillow v12.1.0
│ ├── pyparsing v3.3.2
│ └── python-dateutil v2.9.0.post0
│ └── six v1.17.0
├── numpy v2.4.1
├── pandas v3.0.0
│ ├── numpy v2.4.1
│ └── python-dateutil v2.9.0.post0 (*)
├── pydantic v1.10.26
│ └── typing-extensions v4.15.0
├── scikit-learn v1.8.0
│ ├── joblib v1.5.3
│ ├── numpy v2.4.1
│ ├── scipy v1.17.0
│ │ └── numpy v2.4.1
│ └── threadpoolctl v3.6.0
├── pytest v9.0.2 (group: dev)
│ ├── iniconfig v2.3.0
│ ├── packaging v25.0
│ ├── pluggy v1.6.0
│ └── pygments v2.19.2
└── ruff v0.14.13 (group: dev)
(*) Package tree already displayed
New project setup:
calculate_ci.py
The script above computes a confidence interval but requires numpy and scipy.
Sta 663 - Spring 2026