Python Workshop - SENCE

Infos

Python

Python is an interpreted, high-level, general-purpose programming language. Created by Guido van Rossum and first released in 1991, Python's design philosophy emphasizes code readability with its notable use of significant whitespace. Its language constructs and object-oriented approach aim to help programmers write clear, logical code for small and large-scale projects.

How to install Python + Libraries

Download and install Anaconda (Python 3.9).

Check if Python has been installed

Start-Menu
"Anaconda Prompt" + Enter ↵
python + Enter ↵
print('Ja') + Enter ↵
exit() + Enter ↵

Launch Jupyter Notebook

Start-Menu
"Jupyter Notebook" + Enter ↵
Click on New (top-right)
Choose Python 3

Anaconda Navigator

Anaconda Navigator is a graphical user interface that is automatically installed with Anaconda. Navigator will open if the installation was successful.

Warning

This interface can also be really slow, and might crash. You don't actually need Anaconda Navigator to launch the listed programs, e.g. jupyter notebook or spyder.

Windows: Click Start, search or select Anaconda Navigator from the menu.
macOS: Click Launchpad, select Anaconda Navigator. Or, use Cmd+Space to open Spotlight Search and type “Navigator” to open the program.
Linux: See next section.

Conda

If you prefer using a command line interface (CLI), you can use conda to verify the installation using Anaconda Prompt on Windows or terminal on Linux and macOS.

To open Anaconda Prompt:

Windows: Click Start, search or select Anaconda Prompt from the menu.
macOS: Cmd+Space to open Spotlight Search and type “Navigator” to open the program.
Linux–CentOS: Open Applications - System Tools - terminal.
Linux–Ubuntu: Open the Dash by clicking the upper left Ubuntu icon, then type “terminal”.

Links

Wichtigste Objekte

Zahlen (int & float)

7
0.001
1 + 1
2 * 3
7 / 2
4 / 2
7 // 2
7 % 2
2**3
11**137

Text (str)

"Hello Python"
len("Hello")
"hello".replace('e', 'a').capitalize()
"1,2;3,4;5,6".replace(',', '.')

Text & Zahlen

2 * 3
'2' * 3
'2*3'
int('2') * 3
float('3.5')
'2' + '3'
'file_%d.txt' % 9
'file_%d.%s' % (3, 'dat')
'a;b;c'.split(';')

Wertlisten (list)

['a', 'b', 'c']
len(['a', 'b', 'c'])
['a', 'b', 'c'][0]
['a', 'b', 'c'][:2]
['a', 'b', 'c'][2:]
['a', 'b', 'c'][::-1]
'abc'[1]
[1, 2, 3] + ['a', 'b', 'c']
';'.join(['a', 'b', 'c'])
range(5)

Boolesche Variable (True & False)

3 == 2
x = 5
y = 4
x > y
x >= x
x != y
12 > 7
'12' > '7'
'art' in 'Stuttgart'
6 in [5, 7, 2, 8, 4, 10, 1, 3, 9]

Mehr Wertlisten

values = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
for value in values:
    print(value)

[2**i for i in values]
[i for i in values if i > 5]
[i for i in values if i % 2 == 0]

Dateien

with open('SchPark01/SchPark_GT_2011_01_01.csv') as csv_file:
    for line in csv_file:
        print(line)

with open('new_file.txt', 'w') as new_file:
    new_file.write("Hello\n")

Verzeichnisse

import glob
glob.glob('SchPark01/*.csv')
glob.glob('SchPark03/*/*/*.csv')

Function

def f(a, b):
    return a + b

f(2, 3)

Sortieren

sorted([3,1,2])
numbers = ['1', '2', '3', '4', '5', '6', '7', '8', '9', '10', '11', '12']
sorted(numbers)
sorted(numbers, key=int)

INSEL

Some examples are included in inselpy_examples

# pip install insel
import insel
insel.block('pi')
insel.block('sum', 2, 3)
insel.block('do', parameters=[1, 10, 1])
insel.template('a_times_b', a=7, b=3)

Übungen

Ziel

Ein Sensor hat vielen Dateien geliefert (365 pro Jahr). Es wäre schön, aus 365 Tagesdateien eine einzige Jahresdatei automatisch zu erzeugen.

Hier sind die Dateien : python_workshop_examples.zip

Die Skriptdateien sind schon da, sie haben aber nur eine Beschreibung und keinen Inhalt.

01_workshop_example.py

# Im SchPark01 :
# Eine Datei pro Tag
# Ohne Header
# Datum; Zeit; Horizontale Strahlung; Umgebungstemperatur
# YYYY/MM/DD;HH:MM;W/m2;Celsius
# 1 Verzeichnis pro Jahr
#
# -> 1 Datei fuers Jahr

02_workshop_example.py

# Im SchPark02 :
# Eine Datei pro Tag
# Mit Header
# Datum; Zeit; Horizontale Strahlung; Umgebungstemperatur
# YYYY/MM/DD;HH:MM;W/m2;Celsius
# 1 Verzeichnis pro Jahr
#
# -> 1 Datei fuers Jahr

03_workshop_example.py

# Im SchPark03 :
# Eine Datei pro Tag
# Mit Header
# Datum; Zeit; Horizontale Strahlung; Umgebungstemperatur
# YYYY/MM/DD;HH:MM;W/m2;Celsius
# 1 Verzeichnis pro Monat
#
# -> 1 Datei fuers Jahr

04_workshop_example.py

# Im SchPark04 :
# Eine Datei pro Tag
# Mit Header
# Datum, Zeit, Horizontale Strahlung, Umgebungstemperatur
# YYYY/MM/DD,HH:MM,W/m2,Celsius
# 1 Verzeichnis pro Monat
#
# -> 1 Datei fuers Jahr

05_workshop_example.py

# Im SchPark05 :
# Eine Datei pro Tag
# Mit Header
# Datum; Zeit; Horizontale Strahlung; Umgebungstemperatur
# YYYY/MM/DD;HH:MM;W/m2;Celsius
# 1 Verzeichnis pro Monat
#
# -> 1 Datei fuers Jahr (Excel .CSV)

06_workshop_example.py

# Im SchPark06 :
# Eine Datei pro Tag
# Mit Header
# Zeit; Horizontale Strahlung; Umgebungstemperatur
# HH:MM;W/m2;Celsius
# 1 Verzeichnis pro Monat
#
# -> 1 Datei fuers Jahr

07_workshop_example.py

# Im SchPark07 :
# Eine Datei pro Tag (Tag_Monat_Jahr.csv)
# Mit Header
# Zeit; Horizontale Strahlung; Umgebungstemperatur
# HH:MM;W/m2;Celsius
# 1 Verzeichnis pro Jahr
#
# -> 1 Datei fuers Jahr

08_workshop_example.py

# Im SchPark08 :
# Eine Datei pro Tag (Tag_Monat_Jahr.csv)
# Mit Header
# Zeit; Horizontale Strahlung; Umgebungstemperatur
# HH:MM;W/m2;Celsius
# 1 Verzeichnis fuer 2010 & 2011
#
# -> 1 Datei pro Jahr

Wichtige Libraries

Numpy

Array

import numpy as np

x = np.arange(10)
x+1
(x+1)**2
np.sin(x)
x > 3

2-D Arrays

table = np.arange(50).reshape(10,5)
table**2

Matrix

a = np.mat([[4, 3], [2, 1]])
b = np.mat([[1, 2], [3, 4]])
a*b

Matplotlib

Plot

import matplotlib.pyplot as plt
import numpy as np

t = np.linspace(0, 2*np.pi, 500)
plt.plot(t, np.sin(t))
plt.show()

Sankey

import matplotlib.pyplot as plt
from matplotlib.sankey import Sankey

s = Sankey()
s.add(flows=[0.7, 0.3,-0.5, -0.5],
  labels=['a', 'b', 'c', 'd'],
  orientations = [1, 1, -1, 0] )
s.finish()
plt.show()

Pandas

import pandas as pd
df = pd.read_csv('SchPark01.csv',
         sep=';',
         header = None,
         names = ['date', 'time', 'Gh', 'Ta'],
         parse_dates = [[0, 1]],
         skipinitialspace=True,
         index_col = 0
       )
df.Gh['2011-07-07 12:30']
df.Ta.mean()
df.plot()

Sympy

import sympy
from sympy.solvers import solve
x = sympy.Symbol('x')
solve(sympy.Eq(x**2, x+1), x)
sympy.expand(x*(x+1)*(x+3))

Optimize

from scipy.optimize import minimize
def f(x):
    return (x[0]+2)**2+(x[1]-3)**2

minimize(f, [0,0])

Uncertainties

# pip install uncertainties
from uncertainties import ufloat, umath
x = ufloat(39.5, 0.5)
x**2
umath.log(x)
y = x + 2
y
y - x

Many others

pvlib (Photovoltaics)
NetworkX (Graph theory)
scikit-learn, TensorFlow and keras (machine learning)
Stable Diffusion (Image generation from text)
Kivy GUI apps for desktop & smartphones
django or flask (Web services)
BeautifulSoup (HTML/XML parser)
PyGame (Video Games)
missingno (Visualization of missing data)
Python for ArcGIS, TRNSYS, DAYSIM, Blender, ...

Python for HfT

Lineare Gleichungssysteme Lösen

import numpy as np
a = np.array([[1,2], [3,2]])
b = np.array([19, 29])
# 1*x0 + 2*x1 = 19
# 3*x0 + 2*x1 = 29
x = np.linalg.solve(a, b)
np.dot(a, x)
np.allclose(np.dot(a,x),b)

Mit komplexen Zahlen rechnen

1 + 2j
complex(1, 2)
z = 1 + 2j
abs(z)
z.real
z.imag
z**3
import cmath
cmath.sin(z)
cmath.exp(z)
cmath.rect(1, cmath.pi/3)

Mehrdimensionale Matrizen

import numpy as np

m = np.arange(12)
m = m.reshape(2,3,2)
m[1]
m[1][2]
m[1][2][0]
m[:,2,:]

def f(i,j,k):
    return (i + 3*j + 5*k)

np.fromfunction(f, (2,2,2))

Daten in 2D darstellen

import matplotlib.pyplot as plt

t = np.linspace(0, 2*np.pi, 500)
plt.plot(t, np.sin(t))
plt.show()

Daten in 3D darstellen

from mpl_toolkits.mplot3d import Axes3D # Needed for 3d plots

ax = plt.axes(projection='3d')
z = np.linspace(0, 1, 100)
x = z * np.sin(20 * z)
y = z * np.cos(20 * z)
ax.scatter(x, y, z, c = x+y)
plt.show()

Animationen (movies)

Tools → Preferences → Ipython Console → Graphics → Graphics Backend → Backend: “automatic”

import numpy as np
import matplotlib.pyplot as plt
import matplotlib.animation as animation

fig, ax = plt.subplots()

x = np.arange(0, 2*np.pi, 0.01)
line, = ax.plot(x, np.sin(x))

def animate(i):
    line.set_ydata(np.sin(x + 2*i*np.pi/100.0) *np.cos(2*i*np.pi/200))  # update the data
    return line,

def init():
    line.set_ydata(np.ma.array(x, mask=True))
    return line,

ani = animation.FuncAnimation(fig, animate, np.arange(1, 200), init_func=init,
              interval=25, blit=True)
plt.show()

Fourier Transformation

import numpy as np
import matplotlib.pyplot as plt

N = 1000
T = 0.01
x = np.linspace(0.0, N*T, N)
y = np.where(abs(x)<=0.5, 1, 0) # Rectangular function
yf = np.fft.fft(y)
xf = np.linspace(0.0, 1.0/(2.0*T), N//2)

fig, ax = plt.subplots()
ax.plot(xf, 2.0/N * yf[:N//2])
plt.show()

Ab- und Aufleiten, Nullstellen

from sympy import *
init_printing() # for pretty printing
x,a  = symbols('x a')
f = sin(sqrt((exp(x)+a)/2))
diff(f,x)
integrate(1/(1+x**2),x)
solve(f,x)
f.subs(x,log(-a))

Multi-plots

import numpy as np
import matplotlib.pyplot as plt

N = 5
x = np.linspace(0, 2 * np.pi, 400)
fig, subplots = plt.subplots(N, N, sharex='col', sharey='row')
for (i, j), subplot in np.ndenumerate(subplots):
    subplot.plot(x, i * np.cos(x**2) + j * np.sin(x))

fig.suptitle("i * cos(x**2) + j * sin(x)")
plt.show()

Numpy¶

In [1]:

import numpy as np

In [2]:

x = np.arange(10)

In [3]:

Out[3]:

array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])

In [4]:

x + 1

Out[4]:

array([ 1,  2,  3,  4,  5,  6,  7,  8,  9, 10])

In [5]:

(x + 1)**2

Out[5]:

array([  1,   4,   9,  16,  25,  36,  49,  64,  81, 100])

In [6]:

np.sin(x)

Out[6]:

array([ 0.        ,  0.84147098,  0.90929743,  0.14112001, -0.7568025 ,
       -0.95892427, -0.2794155 ,  0.6569866 ,  0.98935825,  0.41211849])

In [7]:

np.sin(x).dtype

Out[7]:

dtype('float64')

In [8]:

x > 3

Out[8]:

array([False, False, False, False,  True,  True,  True,  True,  True,
        True])

In [9]:

x[:4]

Out[9]:

array([0, 1, 2, 3])

In [10]:

x[7:]

Out[10]:

array([7, 8, 9])

In [11]:

x[x > 3]

Out[11]:

array([4, 5, 6, 7, 8, 9])

In [12]:

x[(x > 3) & (x < 7)]

Out[12]:

array([4, 5, 6])

In [13]:

x[(x > 7) | (x < 3)]

Out[13]:

array([0, 1, 2, 8, 9])

In [14]:

np.arange(50)

Out[14]:

array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16,
       17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33,
       34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49])

In [15]:

np.arange(50).reshape(10,5)

Out[15]:

array([[ 0,  1,  2,  3,  4],
       [ 5,  6,  7,  8,  9],
       [10, 11, 12, 13, 14],
       [15, 16, 17, 18, 19],
       [20, 21, 22, 23, 24],
       [25, 26, 27, 28, 29],
       [30, 31, 32, 33, 34],
       [35, 36, 37, 38, 39],
       [40, 41, 42, 43, 44],
       [45, 46, 47, 48, 49]])

In [16]:

table = _

In [17]:

table

Out[17]:

array([[ 0,  1,  2,  3,  4],
       [ 5,  6,  7,  8,  9],
       [10, 11, 12, 13, 14],
       [15, 16, 17, 18, 19],
       [20, 21, 22, 23, 24],
       [25, 26, 27, 28, 29],
       [30, 31, 32, 33, 34],
       [35, 36, 37, 38, 39],
       [40, 41, 42, 43, 44],
       [45, 46, 47, 48, 49]])

In [18]:

table**2

Out[18]:

array([[   0,    1,    4,    9,   16],
       [  25,   36,   49,   64,   81],
       [ 100,  121,  144,  169,  196],
       [ 225,  256,  289,  324,  361],
       [ 400,  441,  484,  529,  576],
       [ 625,  676,  729,  784,  841],
       [ 900,  961, 1024, 1089, 1156],
       [1225, 1296, 1369, 1444, 1521],
       [1600, 1681, 1764, 1849, 1936],
       [2025, 2116, 2209, 2304, 2401]])

In [19]:

(table**2)[2:4]

Out[19]:

array([[100, 121, 144, 169, 196],
       [225, 256, 289, 324, 361]])

In [20]:

(table**2)[:,2:4]

Out[20]:

array([[   4,    9],
       [  49,   64],
       [ 144,  169],
       [ 289,  324],
       [ 484,  529],
       [ 729,  784],
       [1024, 1089],
       [1369, 1444],
       [1764, 1849],
       [2209, 2304]])

In [21]:

(table**2)[5:7, 2:4]

Out[21]:

array([[ 729,  784],
       [1024, 1089]])

In [22]:

table ** 2 == 1089

Out[22]:

array([[False, False, False, False, False],
       [False, False, False, False, False],
       [False, False, False, False, False],
       [False, False, False, False, False],
       [False, False, False, False, False],
       [False, False, False, False, False],
       [False, False, False,  True, False],
       [False, False, False, False, False],
       [False, False, False, False, False],
       [False, False, False, False, False]])

In [23]:

[index for (index, value) in np.ndenumerate(table**2) if value == 1089]

Out[23]:

[(6, 3)]

In [24]:

l = list(range(10))

In [25]:

Out[25]:

[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

In [26]:

[i * 2 for i in l if i < 5]

Out[26]:

[0, 2, 4, 6, 8]

In [27]:

table**100

Out[27]:

array([[                   0,                    1,                    0,
        -2984622845537545263,                    0],
       [-3842938066129721103,                    0,  3728452490685454945,
                           0,  6627890308811632801],
       [                   0, -5485011738861510223,                    0,
        -8267457965844590319,                    0],
       [ 6813754833676406721,                    0, -1895149777787118527,
                           0,  8207815121025376913],
       [                   0,  1178643571979107377,                    0,
         5958518545374539809,                    0],
       [-7817535966050405663,                    0,  4157753088978724465,
                           0,  3054346751387081297],
       [                   0,  6365139678740040577,                    0,
        -8877898876394183551,                    0],
       [ 8170176069297290577,                    0, -1901415581956121743,
                           0,  2024094702548431329],
       [                   0,  6476859561917718817,                    0,
         5633018509028505393,                    0],
       [ 3548161959473065873,                    0, -2387541571489615039,
                           0,  1459632558914132161]])

In [28]:

a = np.mat('4 3; 2 1')

In [29]:

b = np.mat('1 2; 3 4')

In [30]:

a**2

Out[30]:

matrix([[22, 15],
        [10,  7]])

In [31]:

Out[31]:

matrix([[4, 3],
        [2, 1]])

In [32]:

a*b

Out[32]:

matrix([[13, 20],
        [ 5,  8]])

In [33]:

(np.arange(4)+1).reshape(2,2)

Out[33]:

array([[1, 2],
       [3, 4]])

In [34]:

np.mat(_)

Out[34]:

matrix([[1, 2],
        [3, 4]])