3rdstage's Wiki
Register
Advertisement

Language

  • PEP (Python Enhancement Proposals)
PEP Title Remarks
PEP 8 Style Guide for Python Code
PEP 3119 Introducing Abstract Base Classes

References

Python 3 API

Module Description Sources Remarks
collections Container datatypes Lib/collections/__init__.py
collections.abc Abstract base classes for containers Lib/_collections_abc.py
functools Higher-order functions and operations on callable objects Lib/functools.py reduce()
statistics Provides functions for calculating mathematical statistics of numeric (Real-valued) data. Lib/statistics.py
random Implements pseudo-random number generators for various distributions. Lib/random.py randrange(), choice(), choices()
datetime Supplies classes for manipulating dates and times. Lib/datetime.py
time Provides various time-related functions.
calendar Allows you to output calendars like the Unix cal program, and provides additional useful functions related to the calendar. Lib/calendar.py
os.path Implements some useful functions on pathnames. Lib/posixpath.py, Lib/ntpath.py
pathlib Offers classes representing filesystem paths with semantics appropriate for different operating systems. Lib/pathlib.py since v3.4
urllib Collects several modules for working with URLs. Lib/urllib/ urllib.request, urllib.error, urllib.parse, urllib.robotparser
urllib.request Defines functions and classes which help in opening URLs (mostly HTTP) in a complex world — basic and digest authentication, redirections, cookies and more. Lib/urllib/request.py
zipfile Provides tools to create, read, write, append, and list a ZIP file. Lib/zipfile.py .ZIP File Format Spec
unittest supports test automation, sharing of setup and shutdown code for tests, aggregation of tests into collections, and independence of the tests from the reporting framework. Lib/unittest/__init__.py

Built-in Functions

Built-in Fn. Description Remarks
len(s) Return the length (the number of items) of an object.
hex(x) Convert an integer number to a lowercase hexadecimal string prefixed with “0x”. int()
int(x, base) Return an integer object constructed from a number or string x hex()
enumerate(iterable, start=0) Return an enumerate object.
map(function, iterable, ...) Return an iterator that applies function to every item of iterable, yielding the results.
filter(function, iterable) Construct an iterator from those elements of iterable for which function returns true.
input() Reads a line from input, converts it to a string (stripping a trailing newline), and returns that.
type(object) Return the type of an object. Reflection
dir([object]) Without arguments, return the list of names in the current local scope. With an argument, attempt to return a list of valid attributes for that object. Reflection

Built-in Types

Category Types Remarks
Numeric Types int, float, complex
Sequence Types list, tuple, range
Text Sequence Type str
Binary Sequence Types bytes, bytearray, memoryview
Set Types set, frozenset
Mapping Types dict
Type Description Literal Remarks
list ordered mutable collection ['cat', 'dog', 'snake', 'fish', 'elephant']
tuple ordered immutable collection ('Mon', 'Tue', 'Wed', 'Thu', 'Fri', 'Sat', 'Sun')
dict collection of key-value pairs {'dog': 'bark', 'cat': 'meow', 'snake': 'hiss'} map, associative array
set unordered mutable collection {1, 2, 3, 5, 7, 11, 13}
Data Type Reference W3Schools Reference W3School Tutorial Remarks
str String Method Python String Methods Python Strings
list More on Lists Python List/Array Methods Python Lists
set Set Types — set, frozenset Python Set Methods
dict Mapping Types — dict Python Dictionary Methods

collections Module

Class Description Methods Remarks
Iterator __next__, __iter__

Python-collections

unittest Module

Method Checks Remarks
assertEqual(a, b) a == b
assertNotEqual(a, b) a != b
assertIs(a, b) a is b
assertIsNot(a, b) a is not b
assertRaise(ex, func, args, kwds)

Readings

Grammar

Datatypes

Container Types

Map, Filter, Reduce

# sum of odd number from 0 to 100 
sum = reduce(lambda p, c: p + c if c % 2 == 1 else p, range(100))

# same summation using list comprehensions
sum2 = sum([a for a in range(100) if a % 2 == 1])

Data Analysis

misc

Frameworks

Django

References

Category Command Description Remarks
Setup startproject
startapp
Migration makemigrations Creates new migrations based on the changes detected to your models.
sqlmigrate Prints the SQL for the named migration.
migrate Synchronizes the database state with the current set of models and migrations.
Run runserver Starts a lightweight development Web server on the local machine.

Readings


Jinja2

  • Readings

pytest

  • http://docs.pytest.org/
  • Desc. : a mature full-featured Python testing tool that helps you write better programs.
  • License :

Libraries

Math

NumPy

  • https://numpy.org/
  • Desc. : The fundamental package for scientific computing with Python
  • Sources :
References
Class Description Remarks
ndarray a (usually fixed-size) multidimensional container of items of the same type and size.
ndarray.ndim                   # int, number of array dimensions
ndarray.shape                  # tuple, tuple of array dimensions
ndarray.dtype                  # dtype object, data-type of the array’s elements
Category Routine Description Remarks
Creation numpy.array() Create an array.
numpy.eye() Return a 2-D array with ones on the diagonal and zeros elsewhere.
Manipulation numpy.reshape() Gives a new shape to an array without changing its data
Statistics numpy.mean() Compute the arithmetic mean along the specified axis.
numpy.median() Compute the median along the specified axis.

SymPy

SciPy

  • https://www.scipy.org/
  • Desc. : a Python-based ecosystem of open-source software for mathematics, science, and engineering.
  • Modules
Module Title Description Remarks
scipy.stats Statistical functions contains a large number of probability distributions as well as a growing library of statistical functions.

Data Analysis

pandas

  • https://pandas.pydata.org/
  • Desc. : a fast, powerful, flexible and easy to use open source data analysis and manipulation tool, built on top of the Python programming language.
General Functions
Function Description Remarks
pandas.read_csv() Read a comma-separated values (csv) file into DataFrame.
pandas.to_datetime() Convert argument to datetime.
pandas.crosstab() Compute a simple cross tabulation of two (or more) factors.
pandas.merge() Merge DataFrame or named Series objects with a database-style join. inner/left outer/right outer/full outer join
pandas.get_dummies() Convert categorical variable into dummy/indicator variables.
pandas.crosstab()
Pandas SQL Remarks
crosstab(index=drink["Attend"], columns="count") select ATTEND, count(*) from DRINK group by ATTEND
crosstab(index=drink["Attend"], columns=drink["Name"]) select ATTNED, NAME, count(*) from DRINK group by ATTEND, NAME
Modules
API Source Description Remarks
Series class Series(base.IndexOpsMixin, NDFrame) One-dimensional ndarray with axis labels (including time series)
DataFrame class DataFrame(NDFrame, OpsMixin) Two-dimensional, size-mutable, potentially heterogeneous tabular data
GroupBy class GroupBy(BaseGroupBy[NDFrameT]) Objects are returned by groupby calls
SeriesGroupBy class SeriesGroupBy(GroupBy[Series])
DataFrameGroupBy class DataFrameGroupBy(GroupBy[DataFrame])
pandas.DataFrame
Category Property Description Remarks
Scheme pandas.DataFrame.shape Return a tuple representing the dimensionality of the DataFrame.
pandas.DataFrame.columns The column labels of the DataFrame.
pandas.DataFrame.dtypes Return the dtypes in the DataFrame.
pandas.DataFrame.info() Prints information about a DataFrame including the index dtype and columns, non-null values and memory usage.
Query pandas.DataFrame.head() Return the first n rows. return DataFrame object
pandas.DataFrame.tail() Return the last n rows. return DataFrame object
pandas.DataFrame.astype() Cast a pandas object to a specified dtype.
Filter pandas.DataFrame.loc() Access a group of rows and columns by label(s) or a boolean array. foo.loc[0:10, ['bar', 'baz']]. return DataFrame, Series, or single value.
pandas.DataFrame.iloc() Purely integer-location based indexing for selection by position. foo.iloc[0:11, 3:8], foo.iloc[[0, 3, 5], [1, 2, 5]]
pandas.DataFrame.at() Access a single value for a row/column label pair.
Manipulate pandas.DataFrame.rename() Alter axes labels.
pandas.DataFrame.sort_values() Sort by the values along either axis
pandas.DataFrame.drop() Remove rows or columns by specifying label names and corresponding axis, or by specifying directly index or column names.
pandas.DataFrame.copy() Make a copy of this object’s indices and data.
pandas.DataFrame.reset_index() Reset the index, or a level of it. inplace, drop
Cleaning pandas.DataFrame.isna() Detect missing values. pandas.DataFrame.isnull()]
pandas.DataFrame.dropna() Remove missing values.
pandas.DataFrame.fillna() Fill NA/NaN values using the specified method. method
Aggregation pandas.DataFrame.groupby() Group DataFrame using a mapper or by a Series of columns. as_index, axis
pandas.DataFrame.any() Return whether any element is True, potentially over an axis.
Statisttics pandas.DataFrame.sum() Return the sum of the values over the requested axis.
pandas.DataFrame.mean() Return the mean of the values over the requested axis. inplace
pandas.DataFrame.std() Return sample standard deviation over requested axis.
pandas.DataFrame.describe() Generate descriptive statistics.
pandas.Series
Property Description Remarks
pandas.Series.unique() Return unique values of Series object.
pandas.Series.value_counts() Return a Series containing counts of unique values.
pandas.Series.sum() Return the sum of the values over the requested axis.
pandas.Series.mean() Return the mean of the values over the requested axis.
pandas.Series.max() Return the maximum of the values over the requested axis.
pandas.Series.rolling() Provide rolling window calculations
pandas.Series.shift() Shift index by desired number of periods with an optional time
Readings

pandas datareader

tsfresh

  • https://github.com/blue-yonder/tsfresh
  • Desc. : Provides systematic time-series feature extraction by combining established algorithms from statistics, time-series analysis, signal processing, and nonlinear dynamics with a robust feature selection algorithm.

Cesium

  • https://cesium-ml.org/
  • Desc. : an end-to-end machine learning platform for time-series, from calculation of features to model-building to predictions.

Visualization

Matplotlib

  • https://matplotlib.org/
  • Desc. : a comprehensive library for creating static, animated, and interactive visualizations in Python.
Module Function Description Remarks
matplotlib.pyplot provides a MATLAB-like way of plotting.
scatter() A scatter plot of y vs. x with varying marker size and/or color.
show() Display all open figures.

Seaborn

Tools

pip

  • https://pip.pypa.io/
  • Desc. : a package management system used to install and manage software packages written in Python.

Readings

Commands
Command Description Remarks
pip help command
pip list -vvv List installed packages, including editables. No name filtering. Use grep pipe.
pip show Show information about one or more installed packages.
pip search [options] <query> Search for PyPI packages whose name or summary contains <query>.
pip install --upgrade [options] <package> Upgrade all specified packages to the newest available version.

pyenv

Readings

Command Description Remarks
pyenv versions Lists all Python versions known to pyenv.
pyenv install --list List the all available versions of Python, including Anaconda, Jython, pypy, and stackless.
pyenv install Install a Python version.

Setuptools

  • https://github.com/pypa/setuptools
  • Desc. : a fully-featured, actively-maintained, and stable library designed to facilitate packaging Python projects.
  • License : MIT License

Readings

PyBuilder

Colab

Advertisement