Python Workshop - SENCE
Infos
Python
Python is an interpreted, high-level, general-purpose programming language. Created by Guido van Rossum and first released in 1991, Python's design philosophy emphasizes code readability with its notable use of significant whitespace. Its language constructs and object-oriented approach aim to help programmers write clear, logical code for small and large-scale projects.
How to install Python + Libraries
Download and install Anaconda (Python 3.9).
Check if Python has been installed
- Start-Menu
- "Anaconda Prompt" + Enter ↵
python+ Enter ↵print('Ja')+ Enter ↵exit()+ Enter ↵
Launch Jupyter Notebook
- Start-Menu
- "Jupyter Notebook" + Enter ↵
- Click on
New(top-right) - Choose
Python 3
Anaconda Navigator
Anaconda Navigator is a graphical user interface that is automatically installed with Anaconda. Navigator will open if the installation was successful.
Warning
This interface can also be really slow, and might crash. You don't actually need Anaconda Navigator to launch the listed programs, e.g. jupyter notebook or spyder.
- Windows: Click Start, search or select Anaconda Navigator from the menu.
- macOS: Click Launchpad, select Anaconda Navigator. Or, use Cmd+Space to open Spotlight Search and type “Navigator” to open the program.
- Linux: See next section.
Conda
If you prefer using a command line interface (CLI), you can use conda to verify the installation using Anaconda Prompt on Windows or terminal on Linux and macOS.
To open Anaconda Prompt:
- Windows: Click Start, search or select Anaconda Prompt from the menu.
- macOS: Cmd+Space to open Spotlight Search and type “Navigator” to open the program.
- Linux–CentOS: Open Applications - System Tools - terminal.
- Linux–Ubuntu: Open the Dash by clicking the upper left Ubuntu icon, then type “terminal”.
Links
Wichtigsten Objekte
Zahlen (int & float)
7
0.001
1 + 1
2 * 3
7 / 2
4 / 2
7 // 2
7 % 2
2**3
11**137
Text (str)
"Hello Python"
len("Hello")
"hello".replace('e', 'a').capitalize()
"1,2;3,4;5,6".replace(',', '.')
Text & Zahlen
2 * 3
'2' * 3
'2*3'
int('2') * 3
float('3.5')
'2' + '3'
'file_%d.txt' % 9
'file_%d.%s' % (3, 'dat')
'a;b;c'.split(';')
Wertlisten (list)
['a', 'b', 'c']
len(['a', 'b', 'c'])
['a', 'b', 'c'][0]
['a', 'b', 'c'][:2]
['a', 'b', 'c'][2:]
['a', 'b', 'c'][::-1]
'abc'[1]
[1, 2, 3] + ['a', 'b', 'c']
';'.join(['a', 'b', 'c'])
range(5)
Boolesche Variable (True & False)
3 == 2
x = 5
y = 4
x > y
x >= x
x != y
12 > 7
'12' > '7'
'art' in 'Stuttgart'
6 in [5, 7, 2, 8, 4, 10, 1, 3, 9]
Mehr Wertlisten
values = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
for value in values:
print(value)
[2**i for i in values]
[i for i in values if i > 5]
[i for i in values if i % 2 == 0]
Dateien
with open('SchPark01/SchPark_GT_2011_01_01.csv') as csv_file:
for line in csv_file:
print(line)
with open('new_file.txt', 'w') as new_file:
new_file.write("Hello\n")
Verzeichnisse
import glob
glob.glob('SchPark01/*.csv')
glob.glob('SchPark03/*/*/*.csv')
Function
def f(a, b):
return a + b
f(2, 3)
Sortieren
sorted([3,1,2])
numbers = ['1', '2', '3', '4', '5', '6', '7', '8', '9', '10', '11', '12']
sorted(numbers)
sorted(numbers, key=int)
INSEL
Some examples are included in inselpy_examples
# pip install inselpy
import insel
insel.block('pi')
insel.block('sum', 2, 3)
insel.block('do', parameters=[1, 10, 1])
insel.template('a_times_b', a=7, b=3)
Übungen
Ziel
Ein Sensor hat vielen Dateien geliefert (365 pro Jahr). Es wäre schön, aus 365 Tagesdateien eine einzige Jahresdatei automatisch zu erzeugen.
Hier sind die Dateien : python_workshop_examples.zip
Die Skriptdateien sind schon da, sie haben aber nur eine Beschreibung und keinen Inhalt.
01_workshop_example.py
# Im SchPark01 :
# Eine Datei pro Tag
# Ohne Header
# Datum; Zeit; Horizontale Strahlung; Umgebungstemperatur
# YYYY/MM/DD;HH:MM;W/m2;Celsius
# 1 Verzeichnis pro Jahr
#
# -> 1 Datei fuers Jahr
02_workshop_example.py
# Im SchPark02 :
# Eine Datei pro Tag
# Mit Header
# Datum; Zeit; Horizontale Strahlung; Umgebungstemperatur
# YYYY/MM/DD;HH:MM;W/m2;Celsius
# 1 Verzeichnis pro Jahr
#
# -> 1 Datei fuers Jahr
03_workshop_example.py
# Im SchPark03 :
# Eine Datei pro Tag
# Mit Header
# Datum; Zeit; Horizontale Strahlung; Umgebungstemperatur
# YYYY/MM/DD;HH:MM;W/m2;Celsius
# 1 Verzeichnis pro Monat
#
# -> 1 Datei fuers Jahr
04_workshop_example.py
# Im SchPark04 :
# Eine Datei pro Tag
# Mit Header
# Datum, Zeit, Horizontale Strahlung, Umgebungstemperatur
# YYYY/MM/DD,HH:MM,W/m2,Celsius
# 1 Verzeichnis pro Monat
#
# -> 1 Datei fuers Jahr
05_workshop_example.py
# Im SchPark05 :
# Eine Datei pro Tag
# Mit Header
# Datum; Zeit; Horizontale Strahlung; Umgebungstemperatur
# YYYY/MM/DD;HH:MM;W/m2;Celsius
# 1 Verzeichnis pro Monat
#
# -> 1 Datei fuers Jahr (Excel .CSV)
06_workshop_example.py
# Im SchPark06 :
# Eine Datei pro Tag
# Mit Header
# Zeit; Horizontale Strahlung; Umgebungstemperatur
# HH:MM;W/m2;Celsius
# 1 Verzeichnis pro Monat
#
# -> 1 Datei fuers Jahr
07_workshop_example.py
# Im SchPark07 :
# Eine Datei pro Tag (Tag_Monat_Jahr.csv)
# Mit Header
# Zeit; Horizontale Strahlung; Umgebungstemperatur
# HH:MM;W/m2;Celsius
# 1 Verzeichnis pro Jahr
#
# -> 1 Datei fuers Jahr
08_workshop_example.py
# Im SchPark08 :
# Eine Datei pro Tag (Tag_Monat_Jahr.csv)
# Mit Header
# Zeit; Horizontale Strahlung; Umgebungstemperatur
# HH:MM;W/m2;Celsius
# 1 Verzeichnis fuer 2010 & 2011
#
# -> 1 Datei pro Jahr
Wichtigen Libraries
Numpy
Array
import numpy as np
x = np.arange(10)
x+1
(x+1)**2
np.sin(x)
x > 3
2-D Arrays
table = np.arange(50).reshape(10,5)
table**2
Matrix
a = np.mat([[4, 3], [2, 1]])
b = np.mat([[1, 2], [3, 4]])
a*b
Matplotlib
Plot
import matplotlib.pyplot as plt
import numpy as np
t = np.linspace(0, 2*np.pi, 500)
plt.plot(t, np.sin(t))
plt.show()
Sankey
import matplotlib.pyplot as plt
from matplotlib.sankey import Sankey
s = Sankey()
s.add(flows=[0.7, 0.3,-0.5, -0.5],
labels=['a', 'b', 'c', 'd'],
orientations = [1, 1, -1, 0] )
s.finish()
plt.show()
Pandas
import pandas as pd
df = pd.read_csv('SchPark01.csv',
sep=';',
header = None,
names = ['date', 'time', 'Gh', 'Ta'],
parse_dates = [[0, 1]],
skipinitialspace=True,
index_col = 0
)
df.Gh['2011-07-07 12:30']
df.Ta.mean()
df.plot()
Sympy
import sympy
from sympy.solvers import solve
x = sympy.Symbol('x')
solve(sympy.Eq(x**2, x+1), x)
sympy.expand(x*(x+1)*(x+3))
Optimize
from scipy.optimize import minimize
def f(x):
return (x[0]+2)**2+(x[1]-3)**2
minimize(f, [0,0])
Uncertainties
# pip install uncertainties
from uncertainties import ufloat, umath
x = ufloat(39.5, 0.5)
x**2
umath.log(x)
y = x + 2
y
y - x
Many others
- pvlib (Photovoltaics)
- NetworkX (Graph theory)
- scikit-learn, TensorFlow and keras (machine learning)
- django (Web services)
- BeautifulSoup (HTML/XML parser)
- PyGame (Video Games)
- Python for ArcGIS, TRNSYS, DAYSIM, Blender, ...
Python for HfT
Lineare Gleichungssysteme Lösen
import numpy as np
a = np.array([[1,2], [3,2]])
b = np.array([19, 29])
# 1*x0 + 2*x1 = 19
# 3*x0 + 2*x1 = 29
x = np.linalg.solve(a, b)
np.dot(a, x)
np.allclose(np.dot(a,x),b)
Mit komplexen Zahlen rechnen
1 + 2j
complex(1, 2)
z = 1 + 2j
abs(z)
z.real
z.imag
z**3
import cmath
cmath.sin(z)
cmath.exp(z)
cmath.rect(1, cmath.pi/3)
Mehrdimensionale Matrizen
import numpy as np
m = np.arange(12)
m = m.reshape(2,3,2)
m[1]
m[1][2]
m[1][2][0]
m[:,2,:]
def f(i,j,k):
return (i + 3*j + 5*k)
np.fromfunction(f, (2,2,2))
Daten in 2D darstellen
import matplotlib.pyplot as plt
t = np.linspace(0, 2*np.pi, 500)
plt.plot(t, np.sin(t))
plt.show()
Daten in 3D darstellen
from mpl_toolkits.mplot3d import Axes3D # Needed for 3d plots
ax = plt.axes(projection='3d')
z = np.linspace(0, 1, 100)
x = z * np.sin(20 * z)
y = z * np.cos(20 * z)
ax.scatter(x, y, z, c = x+y)
plt.show()
Animationen (movies)
Tools → Preferences → Ipython Console → Graphics → Graphics Backend → Backend: “automatic”
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.animation as animation
fig, ax = plt.subplots()
x = np.arange(0, 2*np.pi, 0.01)
line, = ax.plot(x, np.sin(x))
def animate(i):
line.set_ydata(np.sin(x + 2*i*np.pi/100.0) *np.cos(2*i*np.pi/200)) # update the data
return line,
def init():
line.set_ydata(np.ma.array(x, mask=True))
return line,
ani = animation.FuncAnimation(fig, animate, np.arange(1, 200), init_func=init,
interval=25, blit=True)
plt.show()
Fourier Transformation
import numpy as np
import matplotlib.pyplot as plt
N = 1000
T = 0.01
x = np.linspace(0.0, N*T, N)
y = np.where(abs(x)<=0.5, 1, 0) # Rectangular function
yf = np.fft.fft(y)
xf = np.linspace(0.0, 1.0/(2.0*T), N//2)
fig, ax = plt.subplots()
ax.plot(xf, 2.0/N * yf[:N//2])
plt.show()
Ab- und Aufleiten, Nullstellen
from sympy import *
init_printing() # for pretty printing
x,a = symbols('x a')
f = sin(sqrt((exp(x)+a)/2))
diff(f,x)
integrate(1/(1+x**2),x)
solve(f,x)
f.subs(x,log(-a))
Multi-plots
import numpy as np
import matplotlib.pyplot as plt
N = 5
x = np.linspace(0, 2 * np.pi, 400)
fig, subplots = plt.subplots(N, N, sharex='col', sharey='row')
for (i, j), subplot in np.ndenumerate(subplots):
subplot.plot(x, i * np.cos(x**2) + j * np.sin(x))
fig.suptitle("i * cos(x**2) + j * sin(x)")
plt.show()
Numpy¶
In [1]:
import numpy as np
In [2]:
x = np.arange(10)
In [3]:
x
Out[3]:
In [4]:
x + 1
Out[4]:
In [5]:
(x + 1)**2
Out[5]:
In [6]:
np.sin(x)
Out[6]:
In [7]:
np.sin(x).dtype
Out[7]:
In [8]:
x > 3
Out[8]:
In [9]:
x[:4]
Out[9]:
In [10]:
x[7:]
Out[10]:
In [11]:
x[x > 3]
Out[11]:
In [12]:
x[(x > 3) & (x < 7)]
Out[12]:
In [13]:
x[(x > 7) | (x < 3)]
Out[13]:
In [14]:
np.arange(50)
Out[14]:
In [15]:
np.arange(50).reshape(10,5)
Out[15]:
In [16]:
table = _
In [17]:
table
Out[17]:
In [18]:
table**2
Out[18]:
In [19]:
(table**2)[2:4]
Out[19]:
In [20]:
(table**2)[:,2:4]
Out[20]:
In [21]:
(table**2)[5:7, 2:4]
Out[21]:
In [22]:
table ** 2 == 1089
Out[22]:
In [23]:
[index for (index, value) in np.ndenumerate(table**2) if value == 1089]
Out[23]:
In [24]:
l = list(range(10))
In [25]:
l
Out[25]:
In [26]:
[i * 2 for i in l if i < 5]
Out[26]:
In [27]:
table**100
Out[27]:
In [28]:
a = np.mat('4 3; 2 1')
In [29]:
b = np.mat('1 2; 3 4')
In [30]:
a**2
Out[30]:
In [31]:
a
Out[31]:
In [32]:
a*b
Out[32]:
In [33]:
(np.arange(4)+1).reshape(2,2)
Out[33]:
In [34]:
np.mat(_)
Out[34]:
In [1]:
import pandas as pd
In [2]:
df = pd.read_csv('Output/SchPark01.csv',
sep = ';',
na_values = ' ',
names = ['date', 'time', 'ghi', 'ta'],
parse_dates = [[0, 1]],
index_col = 'date_time'
)
df
Out[2]:
In [3]:
import matplotlib.pyplot as plt
In [4]:
plt.rcParams['figure.figsize'] = [14, 7]
In [5]:
df.plot();
In [6]:
df.resample('M').mean().plot();
In [7]:
import seaborn as sns
In [8]:
sns.heatmap(
pd.pivot_table(df, values='ghi', index=df.index.time, columns=df.index.dayofyear),
annot=False);
In [9]:
sns.heatmap(
pd.pivot_table(df, values='ta', index=df.index.time, columns=df.index.dayofyear),
annot=False);
In [10]:
# https://stackoverflow.com/a/16345735/6419007
df2 = df.groupby(lambda x: x.time).fillna(method='ffill')
In [11]:
sns.heatmap(
pd.pivot_table(df2, values='ghi', index=df2.index.time, columns=df2.index.dayofyear),
annot=False);
In [12]:
sns.heatmap(
pd.pivot_table(df2, values='ta', index=df2.index.time, columns=df2.index.dayofyear),
annot=False);
In [13]:
df.ta.mean()
Out[13]:
In [14]:
df2.ta.mean()
Out[14]:
In [15]:
df.sort_values('ghi', ascending=False).ghi.plot(use_index=False);
In [16]:
df.sort_values('ta', ascending=False).ta.plot(use_index=False);
In [17]:
df.plot(x='ghi', y='ta', xlabel='GHI [W/m²]', ylabel='Temperature', kind='scatter')
Out[17]:
In [ ]:
pvlib tutorial¶
Install with conda install -c pvlib pvlib
This page contains introductory examples of pvlib python usage. It is based on the Object Oriented code from https://pvlib-python.readthedocs.io/en/stable/introtutorial.html
The goal is to simulate a small PV system in different locations, and try to predict how much energy it could produce.
The code should be as concise as possible, while still delivering plausible results and taking weather into account.
Uploaded to https://gist.github.com/EricDuminil/f646d406967fe965190d2d3fa58df618 Pinged to https://stackoverflow.com/questions/57682450/importing-non-tmy3-format-weather-data-for-use-in-pvlib-simulation/57826625#57826625
Module import¶
In [1]:
from pvlib.pvsystem import PVSystem, retrieve_sam
from pvlib.location import Location
from pvlib.modelchain import ModelChain
from pvlib.temperature import TEMPERATURE_MODEL_PARAMETERS
from pvlib.iotools import pvgis
In [2]:
# Not required, but recommended.
# It avoids downloading same data over and over again from PVGIS.
# https://pypi.org/project/requests-cache/
# pip install requests-cache
import requests_cache
requests_cache.install_cache('pvgis_requests_cache', backend='sqlite')
Locations, Module & Inverter¶
In [3]:
# latitude, longitude, name , altitude, timezone
coordinates = [( 32.2 , -111.0, 'Tucson, Arizona' , 700, 'Etc/GMT+7'),
( 35.1 , -106.6, 'Albuquerque, New Mexico' , 1500, 'Etc/GMT+7'),
( 37.8 , -122.4, 'San Francisco, California', 10, 'Etc/GMT+8'),
( 52.5 , 13.4, 'Berlin, Germany' , 34, 'Etc/GMT-1'),
(-20.9 , 55.5, 'St-Denis, La Réunion' , 100, 'Etc/GMT-4')]
# Get the module and inverter specifications from SAM (https://github.com/NREL/SAM)
module = retrieve_sam('SandiaMod')['Canadian_Solar_CS5P_220M___2009_']
inverter = retrieve_sam('cecinverter')['ABB__MICRO_0_25_I_OUTD_US_208__208V_']
temp_parameters = TEMPERATURE_MODEL_PARAMETERS['sapm']['open_rack_glass_glass']
Simulation¶
In [4]:
for latitude, longitude, name, altitude, timezone in coordinates:
location = Location(latitude, longitude, name=name,
altitude=altitude, tz=timezone)
# Download weather data from PVGIS server
weather, _, info, _ = pvgis.get_pvgis_tmy(location.latitude,
location.longitude)
# Rename columns from PVGIS TMY in order to define the required data.
weather = weather.rename(columns={'G(h)': 'ghi',
'Gb(n)': 'dni',
'Gd(h)': 'dhi',
'T2m': 'temp_air'
})
# Same logic as orientation_strategy='south_at_latitude_tilt', but might be
# a bit clearer for locations in southern hemishpere.
system = PVSystem(module_parameters=module,
inverter_parameters=inverter,
temperature_model_parameters=temp_parameters,
surface_tilt=abs(latitude),
surface_azimuth=180 if latitude > 0 else 0)
mc = ModelChain(system, location)
mc.run_model(weather)
mount = system.arrays[0].mount
# Reporting
nominal_power = module.Impo * module.Vmpo
annual_energy = mc.results.ac.sum()
specific_yield = annual_energy / nominal_power
global_poa = mc.results.total_irrad.poa_global.sum() / 1000
average_ambient_temperature = weather.temp_air.mean()
performance_ratio = specific_yield / global_poa
weather_source = '%s (%d - %d)' % (info['meteo_data']['radiation_db'],
info['meteo_data']['year_min'],
info['meteo_data']['year_max'])
latitude_NS = '%.1f°%s' % (abs(latitude), 'N' if latitude > 0 else 'S')
longitude_EW = '%.1f°%s' % (abs(longitude), 'E' if longitude > 0 else 'W')
print('## %s (%s %s, %s)' % (name, latitude_NS, longitude_EW, timezone))
print('Nominal power : %.2f kWp' % (nominal_power / 1000))
print('Surface azimuth : %.0f °' % mount.surface_azimuth)
print('Surface tilt : %.0f °' % mount.surface_tilt)
print('Weather data source : %s' % weather_source)
print('Global POA irradiance : %.0f kWh / (m² · y)' % global_poa)
print('Average temperature : %.1f °C' % average_ambient_temperature)
print('Total yield : %.0f kWh / y' % (annual_energy / 1000))
print('Specific yield : %.0f kWh / (kWp · y)' % specific_yield)
print('Performance ratio : %.1f %%' % (performance_ratio * 100))
print()
Detailed pvlib report¶
In [1]:
LATITUDE = 48.77
LONGITUDE = 9.18
LOCATION = 'Stuttgart'
TIMEZONE = 'Etc/GMT-1'
ALTITUDE = 400
ALBEDO = 0.2 # Standard is 0.25. Why?
# -20.9 , 55.5, 'St-Denis, La Réunion' , 100, 'Etc/GMT-4')
In [2]:
AZIMUTH = 180
TILT = 25
In [3]:
#TODO: Get timezone automatically
#TODO: Add requirements.txt
#TODO: Define functions each time, with only the strictly required parameters
Enable caching¶
pip install requests-cacheIn [4]:
# Not required. Avoids downloading same data over and over again:
import requests_cache
requests_cache.install_cache('pvgis_requests_cache', backend='sqlite')
Get weather¶
pip install pvlibIn [5]:
from pvlib.iotools import pvgis
In [6]:
weather, _, info, _ = pvgis.get_pvgis_tmy(LATITUDE, LONGITUDE, map_variables=True)
In [7]:
weather_source = '%s (%d - %d)' % (info['meteo_data']['radiation_db'],
info['meteo_data']['year_min'],
info['meteo_data']['year_max'])
latitude_NS = '%.1f°%s' % (abs(LATITUDE), 'N' if LATITUDE > 0 else 'S')
longitude_EW = '%.1f°%s' % (abs(LONGITUDE), 'E' if LONGITUDE > 0 else 'W')
In [8]:
# Rename columns from PVGIS TMY in order to define the required data.
weather = weather.rename(columns={'G(h)': 'ghi',
'Gb(n)': 'dni',
'Gd(h)': 'dhi',
'T2m': 'temp_air',
'WS10m': 'wind_speed' # Does it make sense to use wind speed from 10m height?
})
weather
Out[8]:
In [9]:
# Force all dates to be from the same year
COERCE_YEAR = 2019
weather.index = weather.index.map(lambda dt: dt.replace(year=COERCE_YEAR))
Check and display weather¶
In [10]:
import matplotlib.pyplot as plt
import matplotlib.ticker as mticker
plt.rcParams['figure.figsize'] = [15, 10]
Ambient temperature¶
In [11]:
weather.temp_air.plot(title='Ambient temperature in %s\n%s' % (LOCATION, weather_source), color='#603a47')
plt.gca().yaxis.set_major_formatter(mticker.FormatStrFormatter('%d °C'))