diff --git a/python_patterns/patterns.ipynb b/python_patterns/patterns.ipynb
index 2a769ba..64c6fe1 100644
--- a/python_patterns/patterns.ipynb
+++ b/python_patterns/patterns.ipynb
@@ -1,1542 +1,1600 @@
{
- "metadata": {
- "name": "",
- "signature": "sha256:714a46a359c5b1c3e7e7bd4d19d73221f9def5bcb806840be82541070041d29e"
- },
- "nbformat": 3,
- "nbformat_minor": 0,
- "worksheets": [
+ "cells": [
{
- "cells": [
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "[Go back](https://github.com/rasbt/python_reference) to the `python_reference` repository."
- ]
- },
- {
- "cell_type": "heading",
- "level": 1,
- "metadata": {},
- "source": [
- "A random collection of useful Python snippets"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "I just cleaned my hard drive and found a couple of useful Python snippets that I had some use for in the past. I thought it would be worthwhile to collect them in a IPython notebook for personal reference and share it with people who might find them useful too. \n",
- "Most of those snippets are hopefully self-explanatory, but I am planning to add more comments and descriptions in future."
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "
\n",
- "
"
- ]
- },
- {
- "cell_type": "heading",
- "level": 2,
- "metadata": {},
- "source": [
- "Table of Contents"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "- [Bitstrings from positive and negative elements in a list](#Bitstrings-from-positive-and-negative-elements-in-a-list)\n",
- "- [Command line arguments 1 - sys.argv](#Command-line-arguments-1---sys.argv)\n",
- "- [Data and time basics](#Data-and-time-basics)\n",
- "- [Differences between 2 files](#Differences-between-2-files)\n",
- "- [Differences between successive elements in a list](#Differences-between-successive-elements-in-a-list)\n",
- "- [Doctest example](#Doctest-example)\n",
- "- [English language detection](#English-language-detection)\n",
- "- [File browsing basics](#File-browsing-basics)\n",
- "- [File reading basics](#File-reading-basics)\n",
- "- [Indices of min and max elements from a list](#Indices-of-min-and-max-elements-from-a-list)\n",
- "- [Lambda functions](#Lambda-functions)\n",
- "- [Private functions](#Private-functions)\n",
- "- [Namedtuples](#Namedtuples)\n",
- "- [Normalizing data](#Normalizing-data)\n",
- "- [NumPy essentials](#NumPy-essentials)\n",
- "- [Pickling Python objects to bitstreams](#Pickling-Python-objects-to-bitstreams)\n",
- "- [Python version check](#Python-version-check)\n",
- "- [Runtime within a script](#Runtime-within-a-script)\n",
- "- [Sorting lists of tuples by elements](#Sorting-lists-of-tuples-by-elements)\n",
- "- [Sorting multiple lists relative to each other](#Sorting-multiple-lists-relative-to-each-other)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "
\n",
- "
"
- ]
- },
- {
- "cell_type": "code",
- "collapsed": false,
- "input": [
- "%load_ext watermark"
- ],
- "language": "python",
- "metadata": {},
- "outputs": [],
- "prompt_number": 1
- },
- {
- "cell_type": "code",
- "collapsed": false,
- "input": [
- "%watermark -d -a \"Sebastian Raschka\" -v"
- ],
- "language": "python",
- "metadata": {},
- "outputs": [
- {
- "output_type": "stream",
- "stream": "stdout",
- "text": [
- "Sebastian Raschka 26/09/2014 \n",
- "\n",
- "CPython 3.4.1\n",
- "IPython 2.0.0\n"
- ]
- }
- ],
- "prompt_number": 2
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "[More information](https://github.com/rasbt/watermark) about the `watermark` magic command extension."
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "
\n",
- "
"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "
\n",
- "
"
- ]
- },
- {
- "cell_type": "heading",
- "level": 2,
- "metadata": {},
- "source": [
- "Bitstrings from positive and negative elements in a list"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "[back to top](#Table-of-Contents)"
- ]
- },
- {
- "cell_type": "code",
- "collapsed": false,
- "input": [
- "# Generating a bitstring from a Python list or numpy array\n",
- "# where all postive values -> 1\n",
- "# all negative values -> 0\n",
- "\n",
- "import numpy as np\n",
- "\n",
- "def make_bitstring(ary):\n",
- " return np.where(ary > 0, 1, 0)\n",
- "\n",
- "\n",
- "def faster_bitstring(ary):\n",
- " return np.where(ary > 0).astype('i1')\n",
- "\n",
- "### Example:\n",
- "\n",
- "ary1 = np.array([1, 2, 0.3, -1, -2])\n",
- "print('input values %s' %ary1)\n",
- "print('bitstring %s' %make_bitstring(ary1))"
- ],
- "language": "python",
- "metadata": {},
- "outputs": [
- {
- "output_type": "stream",
- "stream": "stdout",
- "text": [
- "input values [ 1. 2. 0.3 -1. -2. ]\n",
- "bitstring [1 1 1 0 0]\n"
- ]
- }
- ],
- "prompt_number": 3
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "
\n",
- "
"
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "[Go back](https://github.com/rasbt/python_reference) to the `python_reference` repository."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# A random collection of useful Python snippets"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "I just cleaned my hard drive and found a couple of useful Python snippets that I had some use for in the past. I thought it would be worthwhile to collect them in a IPython notebook for personal reference and share it with people who might find them useful too. \n",
+ "Most of those snippets are hopefully self-explanatory, but I am planning to add more comments and descriptions in future."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "
\n",
+ "
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Table of Contents"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "- [Bitstrings from positive and negative elements in a list](#Bitstrings-from-positive-and-negative-elements-in-a-list)\n",
+ "- [Command line arguments 1 - sys.argv](#Command-line-arguments-1---sys.argv)\n",
+ "- [Data and time basics](#Data-and-time-basics)\n",
+ "- [Differences between 2 files](#Differences-between-2-files)\n",
+ "- [Differences between successive elements in a list](#Differences-between-successive-elements-in-a-list)\n",
+ "- [Doctest example](#Doctest-example)\n",
+ "- [English language detection](#English-language-detection)\n",
+ "- [File browsing basics](#File-browsing-basics)\n",
+ "- [File reading basics](#File-reading-basics)\n",
+ "- [Indices of min and max elements from a list](#Indices-of-min-and-max-elements-from-a-list)\n",
+ "- [Lambda functions](#Lambda-functions)\n",
+ "- [Private functions](#Private-functions)\n",
+ "- [Namedtuples](#Namedtuples)\n",
+ "- [Normalizing data](#Normalizing-data)\n",
+ "- [NumPy essentials](#NumPy-essentials)\n",
+ "- [Pickling Python objects to bitstreams](#Pickling-Python-objects-to-bitstreams)\n",
+ "- [Python version check](#Python-version-check)\n",
+ "- [Runtime within a script](#Runtime-within-a-script)\n",
+ "- [Sorting lists of tuples by elements](#Sorting-lists-of-tuples-by-elements)\n",
+ "- [Sorting multiple lists relative to each other](#Sorting-multiple-lists-relative-to-each-other)\n",
+ "- [Using namedtuples](#Using-namedtuples)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "
\n",
+ "
"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 1,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "%load_ext watermark"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Sebastian Raschka 26/09/2014 \n",
+ "\n",
+ "CPython 3.4.1\n",
+ "IPython 2.0.0\n"
]
- },
- {
- "cell_type": "heading",
- "level": 2,
- "metadata": {},
- "source": [
- "Command line arguments 1 - sys.argv"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "[back to top](#Table-of-Contents)"
- ]
- },
- {
- "cell_type": "code",
- "collapsed": false,
- "input": [
- "%%file cmd_line_args_1_sysarg.py\n",
- "import sys\n",
- "\n",
- "def error(msg):\n",
- " \"\"\"Prints error message, sends it to stderr, and quites the program.\"\"\"\n",
- " sys.exit(msg)\n",
- "\n",
- "args = sys.argv[1:] # sys.argv[0] is the name of the python script itself\n",
- "\n",
- "try:\n",
- " arg1 = int(args[0])\n",
- " arg2 = args[1]\n",
- " arg3 = args[2]\n",
- " print(\"Everything okay!\")\n",
- "\n",
- "except ValueError:\n",
- " error(\"First argument must be integer type!\")\n",
- "\n",
- "except IndexError:\n",
- " error(\"Requires 3 arguments!\")"
- ],
- "language": "python",
- "metadata": {},
- "outputs": [
- {
- "output_type": "stream",
- "stream": "stdout",
- "text": [
- "Overwriting cmd_line_args_1_sysarg.py\n"
- ]
- }
- ],
- "prompt_number": 5
- },
- {
- "cell_type": "code",
- "collapsed": false,
- "input": [
- "% run cmd_line_args_1_sysarg.py 1 2 3"
- ],
- "language": "python",
- "metadata": {},
- "outputs": [
- {
- "output_type": "stream",
- "stream": "stdout",
- "text": [
- "Everything okay!\n"
- ]
- }
- ],
- "prompt_number": 6
- },
- {
- "cell_type": "code",
- "collapsed": false,
- "input": [
- "% run cmd_line_args_1_sysarg.py a 2 3"
- ],
- "language": "python",
- "metadata": {},
- "outputs": [
- {
- "ename": "SystemExit",
- "evalue": "First argument must be integer type!",
- "output_type": "pyerr",
- "traceback": [
- "An exception has occurred, use %tb to see the full traceback.\n",
- "\u001b[0;31mSystemExit\u001b[0m\u001b[0;31m:\u001b[0m First argument must be integer type!\n"
- ]
- }
- ],
- "prompt_number": 7
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "
\n",
- "
"
- ]
- },
- {
- "cell_type": "heading",
- "level": 2,
- "metadata": {},
- "source": [
- "Data and time basics"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "[back to top](#Table-of-Contents)"
- ]
- },
- {
- "cell_type": "code",
- "collapsed": false,
- "input": [
- "import time\n",
- "\n",
- "# print time HOURS:MINUTES:SECONDS\n",
- "# e.g., '10:50:58'\n",
- "print(time.strftime(\"%H:%M:%S\"))\n",
- "\n",
- "# print current date DAY:MONTH:YEAR\n",
- "# e.g., '06/03/2014'\n",
- "print(time.strftime(\"%d/%m/%Y\"))"
- ],
- "language": "python",
- "metadata": {},
- "outputs": [
- {
- "output_type": "stream",
- "stream": "stdout",
- "text": [
- "13:28:05\n",
- "26/09/2014\n"
- ]
- }
- ],
- "prompt_number": 7
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "
\n",
- "
"
- ]
- },
- {
- "cell_type": "heading",
- "level": 2,
- "metadata": {},
- "source": [
- "Differences between 2 files"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "[back to top](#Table-of-Contents)"
- ]
- },
- {
- "cell_type": "code",
- "collapsed": false,
- "input": [
- "%%file id_file1.txt\n",
- "1234\n",
- "2342\n",
- "2341"
- ],
- "language": "python",
- "metadata": {},
- "outputs": [
- {
- "output_type": "stream",
- "stream": "stdout",
- "text": [
- "Writing id_file1.txt\n"
- ]
- }
- ],
- "prompt_number": 9
- },
- {
- "cell_type": "code",
- "collapsed": false,
- "input": [
- "%%file id_file2.txt\n",
- "5234\n",
- "3344\n",
- "2341"
- ],
- "language": "python",
- "metadata": {},
- "outputs": [
- {
- "output_type": "stream",
- "stream": "stdout",
- "text": [
- "Writing id_file2.txt\n"
- ]
- }
- ],
- "prompt_number": 10
- },
- {
- "cell_type": "code",
- "collapsed": false,
- "input": [
- "# Print lines that are different between 2 files. Insensitive\n",
- "# to the order of the file contents.\n",
- "\n",
- "id_set1 = set()\n",
- "id_set2 = set()\n",
- "\n",
- "with open('id_file1.txt', 'r') as id_file:\n",
- " for line in id_file:\n",
- " id_set1.add(line.strip())\n",
- "\n",
- "with open('id_file2.txt', 'r') as id_file:\n",
- " for line in id_file:\n",
- " id_set2.add(line.strip()) \n",
- "\n",
- "diffs = id_set2.difference(id_set1)\n",
- "\n",
- "for d in diffs:\n",
- " print(d)\n",
- "print(\"Total differences:\",len(diffs))"
- ],
- "language": "python",
- "metadata": {},
- "outputs": [
- {
- "output_type": "stream",
- "stream": "stdout",
- "text": [
- "5234\n",
- "3344\n",
- "Total differences: 2\n"
- ]
- }
- ],
- "prompt_number": 11
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "
\n",
- "
"
- ]
- },
- {
- "cell_type": "heading",
- "level": 2,
- "metadata": {},
- "source": [
- "Differences between successive elements in a list"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "[back to top](#Table-of-Contents)"
- ]
- },
- {
- "cell_type": "code",
- "collapsed": false,
- "input": [
- "from itertools import islice\n",
- "\n",
- "lst = [1,2,3,5,8]\n",
- "diff = [j - i for i, j in zip(lst, islice(lst, 1, None))]\n",
- "print(diff)"
- ],
- "language": "python",
- "metadata": {},
- "outputs": [
- {
- "output_type": "stream",
- "stream": "stdout",
- "text": [
- "[1, 1, 2, 3]\n"
- ]
- }
- ],
- "prompt_number": 12
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "
\n",
- "
"
- ]
- },
- {
- "cell_type": "heading",
- "level": 2,
- "metadata": {},
- "source": [
- "Doctest example"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "[back to top](#Table-of-Contents)"
- ]
- },
- {
- "cell_type": "code",
- "collapsed": false,
- "input": [
- "def subtract(a, b):\n",
- " \"\"\"\n",
- " Subtracts second from first number and returns result.\n",
- " >>> subtract(10, 5)\n",
- " 5\n",
- " >>> subtract(11, 0.7)\n",
- " 10.3\n",
- " \"\"\"\n",
- " return a-b\n",
- "\n",
- "if __name__ == \"__main__\": # is 'false' if imported\n",
- " import doctest\n",
- " doctest.testmod()\n",
- " print('ok')"
- ],
- "language": "python",
- "metadata": {},
- "outputs": [
- {
- "output_type": "stream",
- "stream": "stdout",
- "text": [
- "ok\n"
- ]
- }
- ],
- "prompt_number": 17
- },
- {
- "cell_type": "code",
- "collapsed": false,
- "input": [
- "def hello_world():\n",
- " \"\"\"\n",
- " Returns 'Hello, World'\n",
- " >>> hello_world()\n",
- " 'Hello, World'\n",
- " \"\"\"\n",
- " return 'hello world'\n",
- "\n",
- "if __name__ == \"__main__\": # is 'false' if imported\n",
- " import doctest\n",
- " doctest.testmod()"
- ],
- "language": "python",
- "metadata": {},
- "outputs": [
- {
- "output_type": "stream",
- "stream": "stdout",
- "text": [
- "**********************************************************************\n",
- "File \"__main__\", line 4, in __main__.hello_world\n",
- "Failed example:\n",
- " hello_world()\n",
- "Expected:\n",
- " 'Hello, World'\n",
- "Got:\n",
- " 'hello world'\n",
- "**********************************************************************\n",
- "1 items had failures:\n",
- " 1 of 1 in __main__.hello_world\n",
- "***Test Failed*** 1 failures.\n"
- ]
- }
- ],
- "prompt_number": 18
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "
\n",
- "
"
- ]
- },
- {
- "cell_type": "heading",
- "level": 2,
- "metadata": {},
- "source": [
- "English language detection"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "[back to top](#Table-of-Contents)"
- ]
- },
- {
- "cell_type": "code",
- "collapsed": false,
- "input": [
- "import nltk\n",
- "\n",
- "def eng_ratio(text):\n",
- " ''' Returns the ratio of non-English to English words from a text '''\n",
- "\n",
- " english_vocab = set(w.lower() for w in nltk.corpus.words.words()) \n",
- " text_vocab = set(w.lower() for w in text.split() if w.lower().isalpha()) \n",
- " unusual = text_vocab.difference(english_vocab)\n",
- " diff = len(unusual)/len(text_vocab)\n",
- " return diff\n",
- " \n",
- "text = 'This is a test fahrrad'\n",
- "\n",
- "print(eng_ratio(text))"
- ],
- "language": "python",
- "metadata": {},
- "outputs": [
- {
- "output_type": "stream",
- "stream": "stdout",
- "text": [
- "0.2\n"
- ]
- }
- ],
- "prompt_number": 1
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "
\n",
- "
"
- ]
- },
- {
- "cell_type": "heading",
- "level": 2,
- "metadata": {},
- "source": [
- "File browsing basics"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "[back to top](#Table-of-Contents)"
- ]
- },
- {
- "cell_type": "code",
- "collapsed": false,
- "input": [
- "import os\n",
- "import shutil\n",
- "import glob\n",
- "\n",
- "# working directory\n",
- "c_dir = os.getcwd() # show current working directory\n",
- "os.listdir(c_dir) # shows all files in the working directory\n",
- "os.chdir('~/Data') # change working directory\n",
- "\n",
- "\n",
- "# get all files in a directory\n",
- "glob.glob('/Users/sebastian/Desktop/*')\n",
- "\n",
- "# e.g., ['/Users/sebastian/Desktop/untitled folder', '/Users/sebastian/Desktop/Untitled.txt']\n",
- "\n",
- "# walk\n",
- "tree = os.walk(c_dir) \n",
- "# moves through sub directories and creates a 'generator' object of tuples\n",
- "# ('dir', [file1, file2, ...] [subdirectory1, subdirectory2, ...]), \n",
- "# (...), ...\n",
- "\n",
- "#check files: returns either True or False\n",
- "os.exists('../rel_path')\n",
- "os.exists('/home/abs_path')\n",
- "os.isfile('./file.txt')\n",
- "os.isdir('./subdir')\n",
- "\n",
- "\n",
- "# file permission (True or False\n",
- "os.access('./some_file', os.F_OK) # File exists? Python 2.7\n",
- "os.access('./some_file', os.R_OK) # Ok to read? Python 2.7\n",
- "os.access('./some_file', os.W_OK) # Ok to write? Python 2.7\n",
- "os.access('./some_file', os.X_OK) # Ok to execute? Python 2.7\n",
- "os.access('./some_file', os.X_OK | os.W_OK) # Ok to execute or write? Python 2.7\n",
- "\n",
- "# join (creates operating system dependent paths)\n",
- "os.path.join('a', 'b', 'c')\n",
- "# 'a/b/c' on Unix/Linux\n",
- "# 'a\\\\b\\\\c' on Windows\n",
- "os.path.normpath('a/b/c') # converts file separators\n",
- "\n",
- "\n",
- "# os.path: direcory and file names\n",
- "os.path.samefile('./some_file', '/home/some_file') # True if those are the same\n",
- "os.path.dirname('./some_file') # returns '.' (everythin but last component)\n",
- "os.path.basename('./some_file') # returns 'some_file' (only last component\n",
- "os.path.split('./some_file') # returns (dirname, basename) or ('.', 'some_file)\n",
- "os.path.splitext('./some_file.txt') # returns ('./some_file', '.txt')\n",
- "os.path.splitdrive('./some_file.txt') # returns ('', './some_file.txt')\n",
- "os.path.isabs('./some_file.txt') # returns False (not an absolute path)\n",
- "os.path.abspath('./some_file.txt')\n",
- "\n",
- "\n",
- "# create and delete files and directories\n",
- "os.mkdir('./test') # create a new direcotory\n",
- "os.rmdir('./test') # removes an empty direcotory\n",
- "os.removedirs('./test') # removes nested empty directories\n",
- "os.remove('file.txt') # removes an individual file\n",
- "shutil.rmtree('./test') # removes directory (empty or not empty)\n",
- "\n",
- "os.rename('./dir_before', './renamed') # renames directory if destination doesn't exist\n",
- "shutil.move('./dir_before', './renamed') # renames directory always\n",
- "\n",
- "shutil.copytree('./orig', './copy') # copies a directory recursively\n",
- "shutil.copyfile('file', 'copy') # copies a file\n",
- "\n",
- " \n",
- "# Getting files of particular type from directory\n",
- "files = [f for f in os.listdir(s_pdb_dir) if f.endswith(\".txt\")]\n",
- " \n",
- "# Copy and move\n",
- "shutil.copyfile(\"/path/to/file\", \"/path/to/new/file\") \n",
- "shutil.copy(\"/path/to/file\", \"/path/to/directory\")\n",
- "shutil.move(\"/path/to/file\",\"/path/to/directory\")\n",
- " \n",
- "# Check if file or directory exists\n",
- "os.path.exists(\"file or directory\")\n",
- "os.path.isfile(\"file\")\n",
- "os.path.isdir(\"directory\")\n",
- " \n",
- "# Working directory and absolute path to files\n",
- "os.getcwd()\n",
- "os.path.abspath(\"file\")"
- ],
- "language": "python",
- "metadata": {},
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "
\n",
- "
"
- ]
- },
- {
- "cell_type": "heading",
- "level": 2,
- "metadata": {},
- "source": [
- "File reading basics"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "[back to top](#Table-of-Contents)"
- ]
- },
- {
- "cell_type": "code",
- "collapsed": false,
- "input": [
- "# Note: rb opens file in binary mode to avoid issues with Windows systems\n",
- "# where '\\r\\n' is used instead of '\\n' as newline character(s).\n",
- "\n",
- "\n",
- "# A) Reading in Byte chunks\n",
- "reader_a = open(\"file.txt\", \"rb\")\n",
- "chunks = []\n",
- "data = reader_a.read(64) # reads first 64 bytes\n",
- "while data != \"\":\n",
- " chunks.append(data)\n",
- " data = reader_a.read(64)\n",
- "if data:\n",
- " chunks.append(data)\n",
- "print(len(chunks))\n",
- "reader_a.close()\n",
- "\n",
- "\n",
- "# B) Reading whole file at once into a list of lines\n",
- "with open(\"file.txt\", \"rb\") as reader_b: # recommended syntax, auto closes\n",
- " data = reader_b.readlines() # data is assigned a list of lines\n",
- "print(len(data))\n",
- "\n",
- "\n",
- "# C) Reading whole file at once into a string\n",
- "with open(\"file.txt\", \"rb\") as reader_c:\n",
- " data = reader_c.read() # data is assigned a list of lines\n",
- "print(len(data))\n",
- "\n",
- "\n",
- "# D) Reading line by line into a list\n",
- "data = []\n",
- "with open(\"file.txt\", \"rb\") as reader_d:\n",
- " for line in reader_d:\n",
- " data.append(line)\n",
- "print(len(data))\n"
- ],
- "language": "python",
- "metadata": {},
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "
\n",
- "
"
- ]
- },
- {
- "cell_type": "heading",
- "level": 2,
- "metadata": {},
- "source": [
- "Indices of min and max elements from a list"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "[back to top](#Table-of-Contents)"
- ]
- },
- {
- "cell_type": "code",
- "collapsed": false,
- "input": [
- "import operator\n",
- "\n",
- "values = [1, 2, 3, 4, 5]\n",
- "\n",
- "min_index, min_value = min(enumerate(values), key=operator.itemgetter(1))\n",
- "max_index, max_value = max(enumerate(values), key=operator.itemgetter(1))\n",
- "\n",
- "print('min_index:', min_index, 'min_value:', min_value)\n",
- "print('max_index:', max_index, 'max_value:', max_value)"
- ],
- "language": "python",
- "metadata": {},
- "outputs": [
- {
- "output_type": "stream",
- "stream": "stdout",
- "text": [
- "min_index: 0 min_value: 1\n",
- "max_index: 4 max_value: 5\n"
- ]
- }
- ],
- "prompt_number": 19
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "
\n",
- "
"
- ]
- },
- {
- "cell_type": "heading",
- "level": 2,
- "metadata": {},
- "source": [
- "Lambda functions"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "[back to top](#Table-of-Contents)"
- ]
- },
- {
- "cell_type": "code",
- "collapsed": false,
- "input": [
- "# Lambda functions are just a short-hand way or writing\n",
- "# short function definitions\n",
- "\n",
- "def square_root1(x):\n",
- " return x**0.5\n",
- " \n",
- "square_root2 = lambda x: x**0.5\n",
- "\n",
- "assert(square_root1(9) == square_root2(9))"
- ],
- "language": "python",
- "metadata": {},
- "outputs": [],
- "prompt_number": 20
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "
\n",
- "
"
- ]
- },
- {
- "cell_type": "heading",
- "level": 2,
- "metadata": {},
- "source": [
- "Private functions"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "[back to top](#Table-of-Contents)"
- ]
- },
- {
- "cell_type": "code",
- "collapsed": false,
- "input": [
- "def create_message(msg_txt):\n",
- " def _priv_msg(message): # private, no access from outside\n",
- " print(\"{}: {}\".format(msg_txt, message))\n",
- " return _priv_msg # returns a function\n",
- "\n",
- "new_msg = create_message(\"My message\")\n",
- "# note, new_msg is a function\n",
- "\n",
- "new_msg(\"Hello, World\")"
- ],
- "language": "python",
- "metadata": {},
- "outputs": [
- {
- "output_type": "stream",
- "stream": "stdout",
- "text": [
- "My message: Hello, World\n"
- ]
- }
- ],
- "prompt_number": 2
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "
\n",
- "
"
- ]
- },
- {
- "cell_type": "heading",
- "level": 2,
- "metadata": {},
- "source": [
- "Namedtuples"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "[back to top](#Table-of-Contents)"
- ]
- },
- {
- "cell_type": "code",
- "collapsed": false,
- "input": [
- "from collections import namedtuple\n",
- "\n",
- "my_namedtuple = namedtuple('field_name', ['x', 'y', 'z', 'bla', 'blub'])\n",
- "p = my_namedtuple(1, 2, 3, 4, 5)\n",
- "print(p.x, p.y, p.z)"
- ],
- "language": "python",
- "metadata": {},
- "outputs": [
- {
- "output_type": "stream",
- "stream": "stdout",
- "text": [
- "1 2 3\n"
- ]
- }
- ],
- "prompt_number": 25
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "
\n",
- "
"
- ]
- },
- {
- "cell_type": "heading",
- "level": 2,
- "metadata": {},
- "source": [
- "Normalizing data"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "[back to top](#Table-of-Contents)"
- ]
- },
- {
- "cell_type": "code",
- "collapsed": false,
- "input": [
- "def normalize(data, min_val=0, max_val=1):\n",
- " \"\"\"\n",
- " Normalizes values in a list of data points to a range, e.g.,\n",
- " between 0.0 and 1.0. \n",
- " Returns the original object if value is not a integer or float.\n",
- " \n",
- " \"\"\"\n",
- " norm_data = []\n",
- " data_min = min(data)\n",
- " data_max = max(data)\n",
- " for x in data:\n",
- " numerator = x - data_min\n",
- " denominator = data_max - data_min\n",
- " x_norm = (max_val-min_val) * numerator/denominator + min_val\n",
- " norm_data.append(x_norm)\n",
- " return norm_data"
- ],
- "language": "python",
- "metadata": {},
- "outputs": [],
- "prompt_number": 28
- },
- {
- "cell_type": "code",
- "collapsed": false,
- "input": [
- "normalize([1,2,3,4,5])"
- ],
- "language": "python",
- "metadata": {},
- "outputs": [
- {
- "metadata": {},
- "output_type": "pyout",
- "prompt_number": 31,
- "text": [
- "[0.0, 0.25, 0.5, 0.75, 1.0]"
- ]
- }
- ],
- "prompt_number": 31
- },
- {
- "cell_type": "code",
- "collapsed": false,
- "input": [
- "normalize([1,2,3,4,5], min_val=-10, max_val=10)"
- ],
- "language": "python",
- "metadata": {},
- "outputs": [
- {
- "metadata": {},
- "output_type": "pyout",
- "prompt_number": 30,
- "text": [
- "[-10.0, -5.0, 0.0, 5.0, 10.0]"
- ]
- }
- ],
- "prompt_number": 30
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "
\n",
- "
"
- ]
- },
- {
- "cell_type": "heading",
- "level": 2,
- "metadata": {},
- "source": [
- "NumPy essentials"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "[back to top](#Table-of-Contents)"
- ]
- },
- {
- "cell_type": "code",
- "collapsed": false,
- "input": [
- "import numpy as np\n",
- "\n",
- "ary1 = np.array([1,2,3,4,5]) # must be same type\n",
- "ary2 = np.zeros((3,4)) # 3x4 matrix consisiting of 0s \n",
- "ary3 = np.ones((3,4)) # 3x4 matrix consisiting of 1s \n",
- "ary4 = np.identity(3) # 3x3 identity matrix\n",
- "ary5 = ary1.copy() # make a copy of ary1\n",
- "\n",
- "item1 = ary3[0, 0] # item in row1, column1\n",
- "\n",
- "ary2.shape # tuple of dimensions. Here: (3,4)\n",
- "ary2.size # number of elements. Here: 12\n",
- "\n",
- "\n",
- "ary2_t = ary2.transpose() # transposes matrix\n",
- "\n",
- "ary2.ravel() # makes an array linear (1-dimensional)\n",
- " # by concatenating rows\n",
- "ary2.reshape(2,6) # reshapes array (must have same dimensions)\n",
- "\n",
- "ary3[0:2, 0:3] # submatrix of first 2 rows and first 3 columns \n",
- "\n",
- "ary3 = ary3[[2,0,1]] # re-arrange rows\n",
- "\n",
- "\n",
- "# element-wise operations\n",
- "\n",
- "ary1 + ary1\n",
- "ary1 * ary1\n",
- "numpy.dot(ary1, ary1) # matrix/vector (dot) product\n",
- "\n",
- "numpy.sum(ary1, axis=1) # sum of a 1D array, column sums of a 2D array\n",
- "numpy.mean(ary1, axis=1) # mean of a 1D array, column means of a 2D array"
- ],
- "language": "python",
- "metadata": {},
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "
\n",
- "
"
- ]
- },
- {
- "cell_type": "heading",
- "level": 2,
- "metadata": {},
- "source": [
- "Pickling Python objects to bitstreams"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "[back to top](#Table-of-Contents)"
- ]
- },
- {
- "cell_type": "code",
- "collapsed": false,
- "input": [
- "import pickle\n",
- "\n",
- "#### Generate some object\n",
- "my_dict = dict()\n",
- "for i in range(1,10):\n",
- " my_dict[i] = \"some text\"\n",
- "\n",
- "#### Save object to file\n",
- "pickle_out = open('my_file.pkl', 'wb')\n",
- "pickle.dump(my_dict, pickle_out)\n",
- "pickle_out.close()\n",
- "\n",
- "#### Load object from file\n",
- "my_object_file = open('my_file.pkl', 'rb')\n",
- "my_dict = pickle.load(my_object_file)\n",
- "my_object_file.close()\n",
- "\n",
- "print(my_dict)"
- ],
- "language": "python",
- "metadata": {},
- "outputs": [
- {
- "output_type": "stream",
- "stream": "stdout",
- "text": [
- "{1: 'some text', 2: 'some text', 3: 'some text', 4: 'some text', 5: 'some text', 6: 'some text', 7: 'some text', 8: 'some text', 9: 'some text'}\n"
- ]
- }
- ],
- "prompt_number": 35
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "
\n",
- "
"
- ]
- },
- {
- "cell_type": "heading",
- "level": 2,
- "metadata": {},
- "source": [
- "Python version check"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "[back to top](#Table-of-Contents)"
- ]
- },
- {
- "cell_type": "code",
- "collapsed": false,
- "input": [
- "import sys\n",
- "\n",
- "def give_letter(word):\n",
- " for letter in word:\n",
- " yield letter\n",
- "\n",
- "if sys.version_info[0] == 3:\n",
- " print('executed in Python 3.x')\n",
- " test = give_letter('Hello')\n",
- " print(next(test))\n",
- " print('in for-loop:')\n",
- " for l in test:\n",
- " print(l)\n",
- "\n",
- "# if Python 2.x\n",
- "if sys.version_info[0] == 2:\n",
- " print('executed in Python 2.x')\n",
- " test = give_letter('Hello')\n",
- " print(test.next())\n",
- " print('in for-loop:') \n",
- " for l in test:\n",
- " print(l)"
- ],
- "language": "python",
- "metadata": {},
- "outputs": [
- {
- "output_type": "stream",
- "stream": "stdout",
- "text": [
- "executed in Python 3.x\n",
- "H\n",
- "in for-loop:\n",
- "e\n",
- "l\n",
- "l\n",
- "o\n"
- ]
- }
- ],
- "prompt_number": 36
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "
\n",
- "
"
- ]
- },
- {
- "cell_type": "heading",
- "level": 2,
- "metadata": {},
- "source": [
- "Runtime within a script"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "[back to top](#Table-of-Contents)"
- ]
- },
- {
- "cell_type": "code",
- "collapsed": false,
- "input": [
- "import time\n",
- "\n",
- "start_time = time.clock()\n",
- "\n",
- "for i in range(10000000):\n",
- " pass\n",
- "\n",
- "elapsed_time = time.clock() - start_time\n",
- "print(\"Time elapsed: {} seconds\".format(elapsed_time))"
- ],
- "language": "python",
- "metadata": {},
- "outputs": [
- {
- "output_type": "stream",
- "stream": "stdout",
- "text": [
- "Time elapsed: 0.49176900000000057 seconds\n"
- ]
- }
- ],
- "prompt_number": 4
- },
- {
- "cell_type": "code",
- "collapsed": false,
- "input": [
- "import timeit\n",
- "elapsed_time = timeit.timeit('for i in range(10000000): pass', number=1)\n",
- "print(\"Time elapsed: {} seconds\".format(elapsed_time))"
- ],
- "language": "python",
- "metadata": {},
- "outputs": [
- {
- "output_type": "stream",
- "stream": "stdout",
- "text": [
- "Time elapsed: 0.3550995970144868 seconds\n"
- ]
- }
- ],
- "prompt_number": 6
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "
\n",
- "
"
- ]
- },
- {
- "cell_type": "heading",
- "level": 2,
- "metadata": {},
- "source": [
- "Sorting lists of tuples by elements"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "[back to top](#Table-of-Contents)"
- ]
- },
- {
- "cell_type": "code",
- "collapsed": false,
- "input": [
- "# Here, we make use of the \"key\" parameter of the in-built \"sorted()\" function \n",
- "# (also available for the \".sort()\" method), which let's us define a function \n",
- "# that is called on every element that is to be sorted. In this case, our \n",
- "# \"key\"-function is a simple lambda function that returns the last item \n",
- "# from every tuple.\n",
- "\n",
- "a_list = [(1,3,'c'), (2,3,'a'), (3,2,'b'), (2,2,'b')]\n",
- "\n",
- "sorted_list = sorted(a_list, key=lambda e: e[::-1])\n",
- "\n",
- "print(sorted_list)"
- ],
- "language": "python",
- "metadata": {},
- "outputs": [
- {
- "output_type": "stream",
- "stream": "stdout",
- "text": [
- "[(2, 3, 'a'), (2, 2, 'b'), (3, 2, 'b'), (1, 3, 'c')]\n"
- ]
- }
- ],
- "prompt_number": 37
- },
- {
- "cell_type": "code",
- "collapsed": false,
- "input": [
- "# prints [(2, 3, 'a'), (2, 2, 'b'), (3, 2, 'b'), (1, 3, 'c')]\n",
- "\n",
- "# If we are only interesting in sorting the list by the last element\n",
- "# of the tuple and don't care about a \"tie\" situation, we can also use\n",
- "# the index of the tuple item directly instead of reversing the tuple \n",
- "# for efficiency.\n",
- "\n",
- "a_list = [(1,3,'c'), (2,3,'a'), (3,2,'b'), (2,2,'b')]\n",
- "\n",
- "sorted_list = sorted(a_list, key=lambda e: e[-1])\n",
- "\n",
- "print(sorted_list)"
- ],
- "language": "python",
- "metadata": {},
- "outputs": [
- {
- "output_type": "stream",
- "stream": "stdout",
- "text": [
- "[(2, 3, 'a'), (3, 2, 'b'), (2, 2, 'b'), (1, 3, 'c')]\n"
- ]
- }
- ],
- "prompt_number": 38
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "
\n",
- "
"
- ]
- },
- {
- "cell_type": "heading",
- "level": 2,
- "metadata": {},
- "source": [
- "Sorting multiple lists relative to each other"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "[back to top](#Table-of-Contents)"
- ]
- },
- {
- "cell_type": "code",
- "collapsed": false,
- "input": [
- "\"\"\"\n",
- "You have 3 lists that you want to sort \"relative\" to each other,\n",
- "for example, picturing each list as a row in a 3x3 matrix: sort it by columns\n",
- "\n",
- "########################\n",
- "If the input lists are\n",
- "########################\n",
- "\n",
- " list1 = ['c','b','a']\n",
- " list2 = [6,5,4]\n",
- " list3 = ['some-val-associated-with-c','another_val-b','z_another_third_val-a']\n",
- "\n",
- "########################\n",
- "the desired outcome is:\n",
- "########################\n",
- "\n",
- " ['a', 'b', 'c'] \n",
- " [4, 5, 6] \n",
- " ['z_another_third_val-a', 'another_val-b', 'some-val-associated-with-c']\n",
- "\n",
- "########################\n",
- "and NOT:\n",
- "########################\n",
- "\n",
- " ['a', 'b', 'c'] \n",
- " [4, 5, 6] \n",
- " ['another_val-b', 'some-val-associated-with-c', 'z_another_third_val-a']\n",
- "\n",
- "\n",
- "\"\"\"\n",
- "\n",
- "list1 = ['c','b','a']\n",
- "list2 = [6,5,4]\n",
- "list3 = ['some-val-associated-with-c','another_val-b','z_another_third_val-a']\n",
- "\n",
- "print('input values:\\n', list1, list2, list3)\n",
- "\n",
- "list1, list2, list3 = [list(t) for t in zip(*sorted(zip(list1, list2, list3)))]\n",
- "\n",
- "print('\\n\\nsorted output:\\n', list1, list2, list3 )"
- ],
- "language": "python",
- "metadata": {},
- "outputs": [
- {
- "output_type": "stream",
- "stream": "stdout",
- "text": [
- "input values:\n",
- " ['c', 'b', 'a'] [6, 5, 4] ['some-val-associated-with-c', 'another_val-b', 'z_another_third_val-a']\n",
- "\n",
- "\n",
- "sorted output:\n",
- " ['a', 'b', 'c'] [4, 5, 6] ['z_another_third_val-a', 'another_val-b', 'some-val-associated-with-c']\n"
- ]
- }
- ],
- "prompt_number": 49
}
],
- "metadata": {}
+ "source": [
+ "%watermark -d -a \"Sebastian Raschka\" -v"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "[More information](https://github.com/rasbt/watermark) about the `watermark` magic command extension."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "
\n",
+ "
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "
\n",
+ "
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Bitstrings from positive and negative elements in a list"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "[back to top](#Table-of-Contents)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 3,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "input values [ 1. 2. 0.3 -1. -2. ]\n",
+ "bitstring [1 1 1 0 0]\n"
+ ]
+ }
+ ],
+ "source": [
+ "# Generating a bitstring from a Python list or numpy array\n",
+ "# where all postive values -> 1\n",
+ "# all negative values -> 0\n",
+ "\n",
+ "import numpy as np\n",
+ "\n",
+ "def make_bitstring(ary):\n",
+ " return np.where(ary > 0, 1, 0)\n",
+ "\n",
+ "\n",
+ "def faster_bitstring(ary):\n",
+ " return np.where(ary > 0).astype('i1')\n",
+ "\n",
+ "### Example:\n",
+ "\n",
+ "ary1 = np.array([1, 2, 0.3, -1, -2])\n",
+ "print('input values %s' %ary1)\n",
+ "print('bitstring %s' %make_bitstring(ary1))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "
\n",
+ "
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Command line arguments 1 - sys.argv"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "[back to top](#Table-of-Contents)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 5,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Overwriting cmd_line_args_1_sysarg.py\n"
+ ]
+ }
+ ],
+ "source": [
+ "%%file cmd_line_args_1_sysarg.py\n",
+ "import sys\n",
+ "\n",
+ "def error(msg):\n",
+ " \"\"\"Prints error message, sends it to stderr, and quites the program.\"\"\"\n",
+ " sys.exit(msg)\n",
+ "\n",
+ "args = sys.argv[1:] # sys.argv[0] is the name of the python script itself\n",
+ "\n",
+ "try:\n",
+ " arg1 = int(args[0])\n",
+ " arg2 = args[1]\n",
+ " arg3 = args[2]\n",
+ " print(\"Everything okay!\")\n",
+ "\n",
+ "except ValueError:\n",
+ " error(\"First argument must be integer type!\")\n",
+ "\n",
+ "except IndexError:\n",
+ " error(\"Requires 3 arguments!\")"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 6,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Everything okay!\n"
+ ]
+ }
+ ],
+ "source": [
+ "% run cmd_line_args_1_sysarg.py 1 2 3"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 7,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "ename": "SystemExit",
+ "evalue": "First argument must be integer type!",
+ "output_type": "error",
+ "traceback": [
+ "An exception has occurred, use %tb to see the full traceback.\n",
+ "\u001b[0;31mSystemExit\u001b[0m\u001b[0;31m:\u001b[0m First argument must be integer type!\n"
+ ]
+ }
+ ],
+ "source": [
+ "% run cmd_line_args_1_sysarg.py a 2 3"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "
\n",
+ "
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Data and time basics"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "[back to top](#Table-of-Contents)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 7,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "13:28:05\n",
+ "26/09/2014\n"
+ ]
+ }
+ ],
+ "source": [
+ "import time\n",
+ "\n",
+ "# print time HOURS:MINUTES:SECONDS\n",
+ "# e.g., '10:50:58'\n",
+ "print(time.strftime(\"%H:%M:%S\"))\n",
+ "\n",
+ "# print current date DAY:MONTH:YEAR\n",
+ "# e.g., '06/03/2014'\n",
+ "print(time.strftime(\"%d/%m/%Y\"))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "
\n",
+ "
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Differences between 2 files"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "[back to top](#Table-of-Contents)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 9,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Writing id_file1.txt\n"
+ ]
+ }
+ ],
+ "source": [
+ "%%file id_file1.txt\n",
+ "1234\n",
+ "2342\n",
+ "2341"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 10,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Writing id_file2.txt\n"
+ ]
+ }
+ ],
+ "source": [
+ "%%file id_file2.txt\n",
+ "5234\n",
+ "3344\n",
+ "2341"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 11,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "5234\n",
+ "3344\n",
+ "Total differences: 2\n"
+ ]
+ }
+ ],
+ "source": [
+ "# Print lines that are different between 2 files. Insensitive\n",
+ "# to the order of the file contents.\n",
+ "\n",
+ "id_set1 = set()\n",
+ "id_set2 = set()\n",
+ "\n",
+ "with open('id_file1.txt', 'r') as id_file:\n",
+ " for line in id_file:\n",
+ " id_set1.add(line.strip())\n",
+ "\n",
+ "with open('id_file2.txt', 'r') as id_file:\n",
+ " for line in id_file:\n",
+ " id_set2.add(line.strip()) \n",
+ "\n",
+ "diffs = id_set2.difference(id_set1)\n",
+ "\n",
+ "for d in diffs:\n",
+ " print(d)\n",
+ "print(\"Total differences:\",len(diffs))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "
\n",
+ "
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Differences between successive elements in a list"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "[back to top](#Table-of-Contents)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 12,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "[1, 1, 2, 3]\n"
+ ]
+ }
+ ],
+ "source": [
+ "from itertools import islice\n",
+ "\n",
+ "lst = [1,2,3,5,8]\n",
+ "diff = [j - i for i, j in zip(lst, islice(lst, 1, None))]\n",
+ "print(diff)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "
\n",
+ "
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Doctest example"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "[back to top](#Table-of-Contents)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 17,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "ok\n"
+ ]
+ }
+ ],
+ "source": [
+ "def subtract(a, b):\n",
+ " \"\"\"\n",
+ " Subtracts second from first number and returns result.\n",
+ " >>> subtract(10, 5)\n",
+ " 5\n",
+ " >>> subtract(11, 0.7)\n",
+ " 10.3\n",
+ " \"\"\"\n",
+ " return a-b\n",
+ "\n",
+ "if __name__ == \"__main__\": # is 'false' if imported\n",
+ " import doctest\n",
+ " doctest.testmod()\n",
+ " print('ok')"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 18,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "**********************************************************************\n",
+ "File \"__main__\", line 4, in __main__.hello_world\n",
+ "Failed example:\n",
+ " hello_world()\n",
+ "Expected:\n",
+ " 'Hello, World'\n",
+ "Got:\n",
+ " 'hello world'\n",
+ "**********************************************************************\n",
+ "1 items had failures:\n",
+ " 1 of 1 in __main__.hello_world\n",
+ "***Test Failed*** 1 failures.\n"
+ ]
+ }
+ ],
+ "source": [
+ "def hello_world():\n",
+ " \"\"\"\n",
+ " Returns 'Hello, World'\n",
+ " >>> hello_world()\n",
+ " 'Hello, World'\n",
+ " \"\"\"\n",
+ " return 'hello world'\n",
+ "\n",
+ "if __name__ == \"__main__\": # is 'false' if imported\n",
+ " import doctest\n",
+ " doctest.testmod()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "
\n",
+ "
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## English language detection"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "[back to top](#Table-of-Contents)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 1,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "0.2\n"
+ ]
+ }
+ ],
+ "source": [
+ "import nltk\n",
+ "\n",
+ "def eng_ratio(text):\n",
+ " ''' Returns the ratio of non-English to English words from a text '''\n",
+ "\n",
+ " english_vocab = set(w.lower() for w in nltk.corpus.words.words()) \n",
+ " text_vocab = set(w.lower() for w in text.split() if w.lower().isalpha()) \n",
+ " unusual = text_vocab.difference(english_vocab)\n",
+ " diff = len(unusual)/len(text_vocab)\n",
+ " return diff\n",
+ " \n",
+ "text = 'This is a test fahrrad'\n",
+ "\n",
+ "print(eng_ratio(text))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "
\n",
+ "
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## File browsing basics"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "[back to top](#Table-of-Contents)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "import os\n",
+ "import shutil\n",
+ "import glob\n",
+ "\n",
+ "# working directory\n",
+ "c_dir = os.getcwd() # show current working directory\n",
+ "os.listdir(c_dir) # shows all files in the working directory\n",
+ "os.chdir('~/Data') # change working directory\n",
+ "\n",
+ "\n",
+ "# get all files in a directory\n",
+ "glob.glob('/Users/sebastian/Desktop/*')\n",
+ "\n",
+ "# e.g., ['/Users/sebastian/Desktop/untitled folder', '/Users/sebastian/Desktop/Untitled.txt']\n",
+ "\n",
+ "# walk\n",
+ "tree = os.walk(c_dir) \n",
+ "# moves through sub directories and creates a 'generator' object of tuples\n",
+ "# ('dir', [file1, file2, ...] [subdirectory1, subdirectory2, ...]), \n",
+ "# (...), ...\n",
+ "\n",
+ "#check files: returns either True or False\n",
+ "os.exists('../rel_path')\n",
+ "os.exists('/home/abs_path')\n",
+ "os.isfile('./file.txt')\n",
+ "os.isdir('./subdir')\n",
+ "\n",
+ "\n",
+ "# file permission (True or False\n",
+ "os.access('./some_file', os.F_OK) # File exists? Python 2.7\n",
+ "os.access('./some_file', os.R_OK) # Ok to read? Python 2.7\n",
+ "os.access('./some_file', os.W_OK) # Ok to write? Python 2.7\n",
+ "os.access('./some_file', os.X_OK) # Ok to execute? Python 2.7\n",
+ "os.access('./some_file', os.X_OK | os.W_OK) # Ok to execute or write? Python 2.7\n",
+ "\n",
+ "# join (creates operating system dependent paths)\n",
+ "os.path.join('a', 'b', 'c')\n",
+ "# 'a/b/c' on Unix/Linux\n",
+ "# 'a\\\\b\\\\c' on Windows\n",
+ "os.path.normpath('a/b/c') # converts file separators\n",
+ "\n",
+ "\n",
+ "# os.path: direcory and file names\n",
+ "os.path.samefile('./some_file', '/home/some_file') # True if those are the same\n",
+ "os.path.dirname('./some_file') # returns '.' (everythin but last component)\n",
+ "os.path.basename('./some_file') # returns 'some_file' (only last component\n",
+ "os.path.split('./some_file') # returns (dirname, basename) or ('.', 'some_file)\n",
+ "os.path.splitext('./some_file.txt') # returns ('./some_file', '.txt')\n",
+ "os.path.splitdrive('./some_file.txt') # returns ('', './some_file.txt')\n",
+ "os.path.isabs('./some_file.txt') # returns False (not an absolute path)\n",
+ "os.path.abspath('./some_file.txt')\n",
+ "\n",
+ "\n",
+ "# create and delete files and directories\n",
+ "os.mkdir('./test') # create a new direcotory\n",
+ "os.rmdir('./test') # removes an empty direcotory\n",
+ "os.removedirs('./test') # removes nested empty directories\n",
+ "os.remove('file.txt') # removes an individual file\n",
+ "shutil.rmtree('./test') # removes directory (empty or not empty)\n",
+ "\n",
+ "os.rename('./dir_before', './renamed') # renames directory if destination doesn't exist\n",
+ "shutil.move('./dir_before', './renamed') # renames directory always\n",
+ "\n",
+ "shutil.copytree('./orig', './copy') # copies a directory recursively\n",
+ "shutil.copyfile('file', 'copy') # copies a file\n",
+ "\n",
+ " \n",
+ "# Getting files of particular type from directory\n",
+ "files = [f for f in os.listdir(s_pdb_dir) if f.endswith(\".txt\")]\n",
+ " \n",
+ "# Copy and move\n",
+ "shutil.copyfile(\"/path/to/file\", \"/path/to/new/file\") \n",
+ "shutil.copy(\"/path/to/file\", \"/path/to/directory\")\n",
+ "shutil.move(\"/path/to/file\",\"/path/to/directory\")\n",
+ " \n",
+ "# Check if file or directory exists\n",
+ "os.path.exists(\"file or directory\")\n",
+ "os.path.isfile(\"file\")\n",
+ "os.path.isdir(\"directory\")\n",
+ " \n",
+ "# Working directory and absolute path to files\n",
+ "os.getcwd()\n",
+ "os.path.abspath(\"file\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "
\n",
+ "
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## File reading basics"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "[back to top](#Table-of-Contents)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "# Note: rb opens file in binary mode to avoid issues with Windows systems\n",
+ "# where '\\r\\n' is used instead of '\\n' as newline character(s).\n",
+ "\n",
+ "\n",
+ "# A) Reading in Byte chunks\n",
+ "reader_a = open(\"file.txt\", \"rb\")\n",
+ "chunks = []\n",
+ "data = reader_a.read(64) # reads first 64 bytes\n",
+ "while data != \"\":\n",
+ " chunks.append(data)\n",
+ " data = reader_a.read(64)\n",
+ "if data:\n",
+ " chunks.append(data)\n",
+ "print(len(chunks))\n",
+ "reader_a.close()\n",
+ "\n",
+ "\n",
+ "# B) Reading whole file at once into a list of lines\n",
+ "with open(\"file.txt\", \"rb\") as reader_b: # recommended syntax, auto closes\n",
+ " data = reader_b.readlines() # data is assigned a list of lines\n",
+ "print(len(data))\n",
+ "\n",
+ "\n",
+ "# C) Reading whole file at once into a string\n",
+ "with open(\"file.txt\", \"rb\") as reader_c:\n",
+ " data = reader_c.read() # data is assigned a list of lines\n",
+ "print(len(data))\n",
+ "\n",
+ "\n",
+ "# D) Reading line by line into a list\n",
+ "data = []\n",
+ "with open(\"file.txt\", \"rb\") as reader_d:\n",
+ " for line in reader_d:\n",
+ " data.append(line)\n",
+ "print(len(data))\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "
\n",
+ "
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Indices of min and max elements from a list"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "[back to top](#Table-of-Contents)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 19,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "min_index: 0 min_value: 1\n",
+ "max_index: 4 max_value: 5\n"
+ ]
+ }
+ ],
+ "source": [
+ "import operator\n",
+ "\n",
+ "values = [1, 2, 3, 4, 5]\n",
+ "\n",
+ "min_index, min_value = min(enumerate(values), key=operator.itemgetter(1))\n",
+ "max_index, max_value = max(enumerate(values), key=operator.itemgetter(1))\n",
+ "\n",
+ "print('min_index:', min_index, 'min_value:', min_value)\n",
+ "print('max_index:', max_index, 'max_value:', max_value)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "
\n",
+ "
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Lambda functions"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "[back to top](#Table-of-Contents)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 20,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "# Lambda functions are just a short-hand way or writing\n",
+ "# short function definitions\n",
+ "\n",
+ "def square_root1(x):\n",
+ " return x**0.5\n",
+ " \n",
+ "square_root2 = lambda x: x**0.5\n",
+ "\n",
+ "assert(square_root1(9) == square_root2(9))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "
\n",
+ "
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Private functions"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "[back to top](#Table-of-Contents)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "My message: Hello, World\n"
+ ]
+ }
+ ],
+ "source": [
+ "def create_message(msg_txt):\n",
+ " def _priv_msg(message): # private, no access from outside\n",
+ " print(\"{}: {}\".format(msg_txt, message))\n",
+ " return _priv_msg # returns a function\n",
+ "\n",
+ "new_msg = create_message(\"My message\")\n",
+ "# note, new_msg is a function\n",
+ "\n",
+ "new_msg(\"Hello, World\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "
\n",
+ "
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Namedtuples"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "[back to top](#Table-of-Contents)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 25,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "1 2 3\n"
+ ]
+ }
+ ],
+ "source": [
+ "from collections import namedtuple\n",
+ "\n",
+ "my_namedtuple = namedtuple('field_name', ['x', 'y', 'z', 'bla', 'blub'])\n",
+ "p = my_namedtuple(1, 2, 3, 4, 5)\n",
+ "print(p.x, p.y, p.z)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "
\n",
+ "
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Normalizing data"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "[back to top](#Table-of-Contents)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 28,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "def normalize(data, min_val=0, max_val=1):\n",
+ " \"\"\"\n",
+ " Normalizes values in a list of data points to a range, e.g.,\n",
+ " between 0.0 and 1.0. \n",
+ " Returns the original object if value is not a integer or float.\n",
+ " \n",
+ " \"\"\"\n",
+ " norm_data = []\n",
+ " data_min = min(data)\n",
+ " data_max = max(data)\n",
+ " for x in data:\n",
+ " numerator = x - data_min\n",
+ " denominator = data_max - data_min\n",
+ " x_norm = (max_val-min_val) * numerator/denominator + min_val\n",
+ " norm_data.append(x_norm)\n",
+ " return norm_data"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 31,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "[0.0, 0.25, 0.5, 0.75, 1.0]"
+ ]
+ },
+ "execution_count": 31,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "normalize([1,2,3,4,5])"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 30,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "[-10.0, -5.0, 0.0, 5.0, 10.0]"
+ ]
+ },
+ "execution_count": 30,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "normalize([1,2,3,4,5], min_val=-10, max_val=10)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "
\n",
+ "
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## NumPy essentials"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "[back to top](#Table-of-Contents)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "import numpy as np\n",
+ "\n",
+ "ary1 = np.array([1,2,3,4,5]) # must be same type\n",
+ "ary2 = np.zeros((3,4)) # 3x4 matrix consisiting of 0s \n",
+ "ary3 = np.ones((3,4)) # 3x4 matrix consisiting of 1s \n",
+ "ary4 = np.identity(3) # 3x3 identity matrix\n",
+ "ary5 = ary1.copy() # make a copy of ary1\n",
+ "\n",
+ "item1 = ary3[0, 0] # item in row1, column1\n",
+ "\n",
+ "ary2.shape # tuple of dimensions. Here: (3,4)\n",
+ "ary2.size # number of elements. Here: 12\n",
+ "\n",
+ "\n",
+ "ary2_t = ary2.transpose() # transposes matrix\n",
+ "\n",
+ "ary2.ravel() # makes an array linear (1-dimensional)\n",
+ " # by concatenating rows\n",
+ "ary2.reshape(2,6) # reshapes array (must have same dimensions)\n",
+ "\n",
+ "ary3[0:2, 0:3] # submatrix of first 2 rows and first 3 columns \n",
+ "\n",
+ "ary3 = ary3[[2,0,1]] # re-arrange rows\n",
+ "\n",
+ "\n",
+ "# element-wise operations\n",
+ "\n",
+ "ary1 + ary1\n",
+ "ary1 * ary1\n",
+ "numpy.dot(ary1, ary1) # matrix/vector (dot) product\n",
+ "\n",
+ "numpy.sum(ary1, axis=1) # sum of a 1D array, column sums of a 2D array\n",
+ "numpy.mean(ary1, axis=1) # mean of a 1D array, column means of a 2D array"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "
\n",
+ "
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Pickling Python objects to bitstreams"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "[back to top](#Table-of-Contents)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 35,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "{1: 'some text', 2: 'some text', 3: 'some text', 4: 'some text', 5: 'some text', 6: 'some text', 7: 'some text', 8: 'some text', 9: 'some text'}\n"
+ ]
+ }
+ ],
+ "source": [
+ "import pickle\n",
+ "\n",
+ "#### Generate some object\n",
+ "my_dict = dict()\n",
+ "for i in range(1,10):\n",
+ " my_dict[i] = \"some text\"\n",
+ "\n",
+ "#### Save object to file\n",
+ "pickle_out = open('my_file.pkl', 'wb')\n",
+ "pickle.dump(my_dict, pickle_out)\n",
+ "pickle_out.close()\n",
+ "\n",
+ "#### Load object from file\n",
+ "my_object_file = open('my_file.pkl', 'rb')\n",
+ "my_dict = pickle.load(my_object_file)\n",
+ "my_object_file.close()\n",
+ "\n",
+ "print(my_dict)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "
\n",
+ "
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Python version check"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "[back to top](#Table-of-Contents)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 36,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "executed in Python 3.x\n",
+ "H\n",
+ "in for-loop:\n",
+ "e\n",
+ "l\n",
+ "l\n",
+ "o\n"
+ ]
+ }
+ ],
+ "source": [
+ "import sys\n",
+ "\n",
+ "def give_letter(word):\n",
+ " for letter in word:\n",
+ " yield letter\n",
+ "\n",
+ "if sys.version_info[0] == 3:\n",
+ " print('executed in Python 3.x')\n",
+ " test = give_letter('Hello')\n",
+ " print(next(test))\n",
+ " print('in for-loop:')\n",
+ " for l in test:\n",
+ " print(l)\n",
+ "\n",
+ "# if Python 2.x\n",
+ "if sys.version_info[0] == 2:\n",
+ " print('executed in Python 2.x')\n",
+ " test = give_letter('Hello')\n",
+ " print(test.next())\n",
+ " print('in for-loop:') \n",
+ " for l in test:\n",
+ " print(l)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "
\n",
+ "
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Runtime within a script"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "[back to top](#Table-of-Contents)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 4,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Time elapsed: 0.49176900000000057 seconds\n"
+ ]
+ }
+ ],
+ "source": [
+ "import time\n",
+ "\n",
+ "start_time = time.clock()\n",
+ "\n",
+ "for i in range(10000000):\n",
+ " pass\n",
+ "\n",
+ "elapsed_time = time.clock() - start_time\n",
+ "print(\"Time elapsed: {} seconds\".format(elapsed_time))"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 6,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Time elapsed: 0.3550995970144868 seconds\n"
+ ]
+ }
+ ],
+ "source": [
+ "import timeit\n",
+ "elapsed_time = timeit.timeit('for i in range(10000000): pass', number=1)\n",
+ "print(\"Time elapsed: {} seconds\".format(elapsed_time))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "
\n",
+ "
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Sorting lists of tuples by elements"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "[back to top](#Table-of-Contents)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 37,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "[(2, 3, 'a'), (2, 2, 'b'), (3, 2, 'b'), (1, 3, 'c')]\n"
+ ]
+ }
+ ],
+ "source": [
+ "# Here, we make use of the \"key\" parameter of the in-built \"sorted()\" function \n",
+ "# (also available for the \".sort()\" method), which let's us define a function \n",
+ "# that is called on every element that is to be sorted. In this case, our \n",
+ "# \"key\"-function is a simple lambda function that returns the last item \n",
+ "# from every tuple.\n",
+ "\n",
+ "a_list = [(1,3,'c'), (2,3,'a'), (3,2,'b'), (2,2,'b')]\n",
+ "\n",
+ "sorted_list = sorted(a_list, key=lambda e: e[::-1])\n",
+ "\n",
+ "print(sorted_list)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 38,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "[(2, 3, 'a'), (3, 2, 'b'), (2, 2, 'b'), (1, 3, 'c')]\n"
+ ]
+ }
+ ],
+ "source": [
+ "# prints [(2, 3, 'a'), (2, 2, 'b'), (3, 2, 'b'), (1, 3, 'c')]\n",
+ "\n",
+ "# If we are only interesting in sorting the list by the last element\n",
+ "# of the tuple and don't care about a \"tie\" situation, we can also use\n",
+ "# the index of the tuple item directly instead of reversing the tuple \n",
+ "# for efficiency.\n",
+ "\n",
+ "a_list = [(1,3,'c'), (2,3,'a'), (3,2,'b'), (2,2,'b')]\n",
+ "\n",
+ "sorted_list = sorted(a_list, key=lambda e: e[-1])\n",
+ "\n",
+ "print(sorted_list)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "
\n",
+ "
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Sorting multiple lists relative to each other"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "[back to top](#Table-of-Contents)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 49,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "input values:\n",
+ " ['c', 'b', 'a'] [6, 5, 4] ['some-val-associated-with-c', 'another_val-b', 'z_another_third_val-a']\n",
+ "\n",
+ "\n",
+ "sorted output:\n",
+ " ['a', 'b', 'c'] [4, 5, 6] ['z_another_third_val-a', 'another_val-b', 'some-val-associated-with-c']\n"
+ ]
+ }
+ ],
+ "source": [
+ "\"\"\"\n",
+ "You have 3 lists that you want to sort \"relative\" to each other,\n",
+ "for example, picturing each list as a row in a 3x3 matrix: sort it by columns\n",
+ "\n",
+ "########################\n",
+ "If the input lists are\n",
+ "########################\n",
+ "\n",
+ " list1 = ['c','b','a']\n",
+ " list2 = [6,5,4]\n",
+ " list3 = ['some-val-associated-with-c','another_val-b','z_another_third_val-a']\n",
+ "\n",
+ "########################\n",
+ "the desired outcome is:\n",
+ "########################\n",
+ "\n",
+ " ['a', 'b', 'c'] \n",
+ " [4, 5, 6] \n",
+ " ['z_another_third_val-a', 'another_val-b', 'some-val-associated-with-c']\n",
+ "\n",
+ "########################\n",
+ "and NOT:\n",
+ "########################\n",
+ "\n",
+ " ['a', 'b', 'c'] \n",
+ " [4, 5, 6] \n",
+ " ['another_val-b', 'some-val-associated-with-c', 'z_another_third_val-a']\n",
+ "\n",
+ "\n",
+ "\"\"\"\n",
+ "\n",
+ "list1 = ['c','b','a']\n",
+ "list2 = [6,5,4]\n",
+ "list3 = ['some-val-associated-with-c','another_val-b','z_another_third_val-a']\n",
+ "\n",
+ "print('input values:\\n', list1, list2, list3)\n",
+ "\n",
+ "list1, list2, list3 = [list(t) for t in zip(*sorted(zip(list1, list2, list3)))]\n",
+ "\n",
+ "print('\\n\\nsorted output:\\n', list1, list2, list3 )"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "
\n",
+ "
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Using namedtuples"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "[back to top](#Table-of-Contents)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "`namedtuples` are high-performance container datatypes in the [`collection`](https://docs.python.org/2/library/collections.html) module (part of Python's stdlib since 2.6).\n",
+ "`namedtuple()` is factory function for creating tuple subclasses with named fields."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 1,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "X-coordinate: 1\n"
+ ]
+ }
+ ],
+ "source": [
+ "from collections import namedtuple\n",
+ "\n",
+ "Coordinates = namedtuple('Coordinates', ['x', 'y', 'z'])\n",
+ "point1 = Coordinates(1, 2, 3)\n",
+ "print('X-coordinate: %d' % point1.x)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
+ "source": []
}
- ]
-}
\ No newline at end of file
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.4.3"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 0
+}