SEAI 2022 - Python - Lab 4

SEAI 2022 - Python - Lab 4

Spatial data and W matrices

Vincenzo Nardelli - vincnardelli@gmail.com - https://github.com/vincnardelli

Before to start… install Python Libraries!

! pip install pysal
! pip install cartoframes 
! pip install --user urllib3==1.22

Collecting pysal
  Downloading https://files.pythonhosted.org/packages/c0/3e/179620647a54eef9ef8173ee82a42b0e33b1e80bbce21998ba8ffc62a0f7/pysal-2.4.0.tar.gz
Collecting libpysal>=4.4.0
[?25l  Downloading https://files.pythonhosted.org/packages/cc/e9/36be6aeea8b6d6ac797c656e1f80bdaddc075d049433495680432d9aedd9/libpysal-4.4.0-py3-none-any.whl (2.4MB)
     |████████████████████████████████| 2.4MB 11.2MB/s 
[?25hCollecting access>=1.1.3
  Downloading https://files.pythonhosted.org/packages/41/50/d7fb5466976af95e4c3481f8408f29744e9411e46a3f9d610a95e741f57b/access-1.1.3-py3-none-any.whl
Collecting esda>=2.3.6
[?25l  Downloading https://files.pythonhosted.org/packages/20/02/7c36f3c7c22f048f50ada6402ff3fbe51e0c5365f7d0227196b84d43c82b/esda-2.3.6-py3-none-any.whl (102kB)
     |████████████████████████████████| 112kB 39.5MB/s 
[?25hCollecting giddy>=2.3.3
[?25l  Downloading https://files.pythonhosted.org/packages/5c/71/f7fc2b41e36b2a06bac30937435d2dedd125f74f3157213ec121ff52d961/giddy-2.3.3-py3-none-any.whl (60kB)
     |████████████████████████████████| 61kB 6.7MB/s 
[?25hCollecting inequality>=1.0.0
  Downloading https://files.pythonhosted.org/packages/74/0f/9ed2d097f29160d0c873f33ffc0b9806c1083e3611acb2143eb66adcf580/inequality-1.0.0.tar.gz
Collecting pointpats>=2.2.0
[?25l  Downloading https://files.pythonhosted.org/packages/5f/2b/a3e99c42b37c3e848eb8e55f2ee2b2d281ae0d49df261f13eaaeba7171a5/pointpats-2.2.0.tar.gz (55kB)
     |████████████████████████████████| 61kB 7.2MB/s 
[?25hCollecting segregation>=1.5.0
[?25l  Downloading https://files.pythonhosted.org/packages/21/82/db8d8f2c25d52e774dcdc65861853f87566c7512204764d1b34acf2d7e50/segregation-1.5.0-py3-none-any.whl (82kB)
     |████████████████████████████████| 92kB 9.8MB/s 
[?25hCollecting spaghetti>=1.5.6
[?25l  Downloading https://files.pythonhosted.org/packages/55/c6/eda10343af40be748218b3e4a8502e5fbf4e882eef8cba8025ada7cdd313/spaghetti-1.5.6-py3-none-any.whl (43kB)
     |████████████████████████████████| 51kB 6.3MB/s 
[?25hCollecting mgwr>=2.1.2
[?25l  Downloading https://files.pythonhosted.org/packages/8e/14/e18342b62655ec7f37624ff936d6f273f82b51913cd558110fd4d670794e/mgwr-2.1.2.tar.gz (41kB)
     |████████████████████████████████| 51kB 5.8MB/s 
[?25hCollecting spglm>=1.0.8
  Downloading https://files.pythonhosted.org/packages/07/8f/03f07807967595d2632dbe0f7c442511a5ff422103b49c76557f9eed40c5/spglm-1.0.8.tar.gz
Collecting spint>=1.0.7
  Downloading https://files.pythonhosted.org/packages/7e/c5/e4862ab3f745a1886135b07e498e77504c14e468d0a6b89f7270f5def979/spint-1.0.7.tar.gz
Collecting spreg>=1.2.2
[?25l  Downloading https://files.pythonhosted.org/packages/40/2d/b6559677b66b2f228fdf02fdb5e7da3466057f9e2c696bea7704138ae346/spreg-1.2.2-py3-none-any.whl (209kB)
     |████████████████████████████████| 215kB 35.8MB/s 
[?25hCollecting spvcm>=0.3.0
[?25l  Downloading https://files.pythonhosted.org/packages/b4/c9/43fb98bc60728b76fceee175119a1ff7e5033ed9012d07570e34a2a19a8f/spvcm-0.3.0.tar.gz (5.7MB)
     |████████████████████████████████| 5.7MB 15.2MB/s 
[?25hCollecting tobler>=0.6.0
  Downloading https://files.pythonhosted.org/packages/5c/88/93acf0ad7c724565975924649928ea9db00e8d94279eb40c7819708b6105/tobler-0.7.0-py3-none-any.whl
Collecting mapclassify>=2.4.2
  Downloading https://files.pythonhosted.org/packages/22/8e/d968c0945d41bb02de0efaa92e31e43a817dc52d30e82b4dfdda407a1903/mapclassify-2.4.2-py3-none-any.whl
Collecting splot>=1.1.3
  Downloading https://files.pythonhosted.org/packages/83/1c/afb3e3eeeda4eef065f0b3249ca74cbcc44af3bc59ddb5a21b5445753955/splot-1.1.3.tar.gz
Collecting spopt>=0.1.1
[?25l  Downloading https://files.pythonhosted.org/packages/a9/df/c8ce65a07f7ab1e69b721533b950dd9854f2f7e6a472eca60c13f5770e24/spopt-0.1.1-py3-none-any.whl (54kB)
     |████████████████████████████████| 61kB 6.6MB/s 
[?25hCollecting urllib3>=1.26
[?25l  Downloading https://files.pythonhosted.org/packages/0c/cd/1e2ec680ec7b09846dc6e605f5a7709dfb9d7128e51a026e7154e18a234e/urllib3-1.26.5-py2.py3-none-any.whl (138kB)
     |████████████████████████████████| 143kB 44.6MB/s 
[?25hCollecting python-dateutil<=2.8.0
[?25l  Downloading https://files.pythonhosted.org/packages/41/17/c62faccbfbd163c7f57f3844689e3a78bae1f403648a6afb1d0866d87fbb/python_dateutil-2.8.0-py2.py3-none-any.whl (226kB)
     |████████████████████████████████| 235kB 44.4MB/s 
[?25hRequirement already satisfied: pytest in /usr/local/lib/python3.7/dist-packages (from pysal) (3.6.4)
Collecting pytest-cov
  Downloading https://files.pythonhosted.org/packages/ba/84/576b071aef9ac9301e5c0ff35d117e12db50b87da6f12e745e9c5f745cc2/pytest_cov-2.12.1-py2.py3-none-any.whl
Requirement already satisfied: coverage in /usr/local/lib/python3.7/dist-packages (from pysal) (3.7.1)
Requirement already satisfied: scipy>=0.11 in /usr/local/lib/python3.7/dist-packages (from libpysal>=4.4.0->pysal) (1.4.1)
Requirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from libpysal>=4.4.0->pysal) (2.23.0)
Requirement already satisfied: numpy>=1.3 in /usr/local/lib/python3.7/dist-packages (from libpysal>=4.4.0->pysal) (1.19.5)
Requirement already satisfied: beautifulsoup4 in /usr/local/lib/python3.7/dist-packages (from libpysal>=4.4.0->pysal) (4.6.3)
Requirement already satisfied: pandas in /usr/local/lib/python3.7/dist-packages (from libpysal>=4.4.0->pysal) (1.1.5)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.7/dist-packages (from libpysal>=4.4.0->pysal) (2.11.3)
Requirement already satisfied: scikit-learn in /usr/local/lib/python3.7/dist-packages (from esda>=2.3.6->pysal) (0.22.2.post1)
Collecting quantecon>=0.4.7
[?25l  Downloading https://files.pythonhosted.org/packages/7e/6b/e78ec352947208c6172e50dcb086f95f7b63eb5204f26f46b676ed5bb31c/quantecon-0.5.0-py3-none-any.whl (229kB)
     |████████████████████████████████| 235kB 47.8MB/s 
[?25hRequirement already satisfied: matplotlib in /usr/local/lib/python3.7/dist-packages (from pointpats>=2.2.0->pysal) (3.2.2)
Collecting opencv-contrib-python>=4.2.0
[?25l  Downloading https://files.pythonhosted.org/packages/43/77/347867aff4a9a5b56ea587a50762fb0ba40b27bb4e8f8f27c25ba57ed807/opencv_contrib_python-4.5.2.52-cp37-cp37m-manylinux2014_x86_64.whl (57.4MB)
     |████████████████████████████████| 57.4MB 66kB/s 
[?25hRequirement already satisfied: seaborn in /usr/local/lib/python3.7/dist-packages (from segregation>=1.5.0->pysal) (0.11.1)
Collecting geopandas
[?25l  Downloading https://files.pythonhosted.org/packages/d7/bf/e9cefb69d39155d122b6ddca53893b61535fa6ffdad70bf5ef708977f53f/geopandas-0.9.0-py2.py3-none-any.whl (994kB)
     |████████████████████████████████| 1.0MB 35.2MB/s 
[?25hRequirement already satisfied: tqdm in /usr/local/lib/python3.7/dist-packages (from segregation>=1.5.0->pysal) (4.41.1)
Collecting rtree
[?25l  Downloading https://files.pythonhosted.org/packages/51/05/5a67111cee91d2165a2bcb855f442186e3d76ddef834596cc84d4875c401/Rtree-0.9.7-cp37-cp37m-manylinux2010_x86_64.whl (994kB)
     |████████████████████████████████| 1.0MB 30.8MB/s 
[?25hRequirement already satisfied: statsmodels in /usr/local/lib/python3.7/dist-packages (from tobler>=0.6.0->pysal) (0.10.2)
Collecting rasterio
[?25l  Downloading https://files.pythonhosted.org/packages/0e/78/c69f7457b7dad6163abde2772abd4c8c0c6498d2ab9fd3f3b0eb73b40951/rasterio-1.2.4-cp37-cp37m-manylinux1_x86_64.whl (19.3MB)
     |████████████████████████████████| 19.3MB 245kB/s 
[?25hCollecting pygeos
[?25l  Downloading https://files.pythonhosted.org/packages/2a/c0/c3c0ebaa718166af6ff03a4e575d4d1ff4ffbd3ccb658be4285fbeaf3c89/pygeos-0.10-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (2.0MB)
     |████████████████████████████████| 2.0MB 25.3MB/s 
[?25hCollecting rasterstats
  Downloading https://files.pythonhosted.org/packages/9f/52/055b2b736e4aa1126c4619a561b44c3bc30fbe48025e6f3275b92928a0a0/rasterstats-0.15.0-py3-none-any.whl
Requirement already satisfied: joblib in /usr/local/lib/python3.7/dist-packages (from tobler>=0.6.0->pysal) (1.0.1)
Requirement already satisfied: networkx in /usr/local/lib/python3.7/dist-packages (from mapclassify>=2.4.2->pysal) (2.5.1)
Requirement already satisfied: descartes in /usr/local/lib/python3.7/dist-packages (from splot>=1.1.3->pysal) (1.1.0)
Collecting pulp
[?25l  Downloading https://files.pythonhosted.org/packages/14/c4/0eec14a0123209c261de6ff154ef3be5cad3fd557c084f468356662e0585/PuLP-2.4-py3-none-any.whl (40.6MB)
     |████████████████████████████████| 40.6MB 98kB/s 
[?25hRequirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/dist-packages (from python-dateutil<=2.8.0->pysal) (1.15.0)
Requirement already satisfied: setuptools in /usr/local/lib/python3.7/dist-packages (from pytest->pysal) (56.1.0)
Requirement already satisfied: atomicwrites>=1.0 in /usr/local/lib/python3.7/dist-packages (from pytest->pysal) (1.4.0)
Requirement already satisfied: pluggy<0.8,>=0.5 in /usr/local/lib/python3.7/dist-packages (from pytest->pysal) (0.7.1)
Requirement already satisfied: py>=1.5.0 in /usr/local/lib/python3.7/dist-packages (from pytest->pysal) (1.10.0)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.7/dist-packages (from pytest->pysal) (21.2.0)
Requirement already satisfied: more-itertools>=4.0.0 in /usr/local/lib/python3.7/dist-packages (from pytest->pysal) (8.7.0)
Requirement already satisfied: toml in /usr/local/lib/python3.7/dist-packages (from pytest-cov->pysal) (0.10.2)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests->libpysal>=4.4.0->pysal) (2020.12.5)
Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->libpysal>=4.4.0->pysal) (2.10)
Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests->libpysal>=4.4.0->pysal) (3.0.4)
Requirement already satisfied: pytz>=2017.2 in /usr/local/lib/python3.7/dist-packages (from pandas->libpysal>=4.4.0->pysal) (2018.9)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python3.7/dist-packages (from jinja2->libpysal>=4.4.0->pysal) (2.0.1)
Requirement already satisfied: sympy in /usr/local/lib/python3.7/dist-packages (from quantecon>=0.4.7->giddy>=2.3.3->pysal) (1.7.1)
Requirement already satisfied: numba>=0.38 in /usr/local/lib/python3.7/dist-packages (from quantecon>=0.4.7->giddy>=2.3.3->pysal) (0.51.2)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->pointpats>=2.2.0->pysal) (2.4.7)
Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->pointpats>=2.2.0->pysal) (1.3.1)
Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.7/dist-packages (from matplotlib->pointpats>=2.2.0->pysal) (0.10.0)
Collecting fiona>=1.8
[?25l  Downloading https://files.pythonhosted.org/packages/9c/fc/9807326c37a6bfb2393ae3e1cca147aa74844562c4d5daa782d6e97ad2bc/Fiona-1.8.20-cp37-cp37m-manylinux1_x86_64.whl (15.4MB)
     |████████████████████████████████| 15.4MB 283kB/s 
[?25hRequirement already satisfied: shapely>=1.6 in /usr/local/lib/python3.7/dist-packages (from geopandas->segregation>=1.5.0->pysal) (1.7.1)
Collecting pyproj>=2.2.0
[?25l  Downloading https://files.pythonhosted.org/packages/11/1d/1c54c672c2faf08d28fe78e15d664c048f786225bef95ad87b6c435cf69e/pyproj-3.1.0-cp37-cp37m-manylinux2010_x86_64.whl (6.6MB)
     |████████████████████████████████| 6.6MB 23.6MB/s 
[?25hRequirement already satisfied: patsy>=0.4.0 in /usr/local/lib/python3.7/dist-packages (from statsmodels->tobler>=0.6.0->pysal) (0.5.1)
Collecting click-plugins
  Downloading https://files.pythonhosted.org/packages/e9/da/824b92d9942f4e472702488857914bdd50f73021efea15b4cad9aca8ecef/click_plugins-1.1.1-py2.py3-none-any.whl
Collecting snuggs>=1.4.1
  Downloading https://files.pythonhosted.org/packages/cc/0e/d27d6e806d6c0d1a2cfdc5d1f088e42339a0a54a09c3343f7f81ec8947ea/snuggs-1.4.7-py3-none-any.whl
Requirement already satisfied: click>=4.0 in /usr/local/lib/python3.7/dist-packages (from rasterio->tobler>=0.6.0->pysal) (7.1.2)
Collecting cligj>=0.5
  Downloading https://files.pythonhosted.org/packages/73/86/43fa9f15c5b9fb6e82620428827cd3c284aa933431405d1bcf5231ae3d3e/cligj-0.7.2-py3-none-any.whl
Collecting affine
  Downloading https://files.pythonhosted.org/packages/ac/a6/1a39a1ede71210e3ddaf623982b06ecfc5c5c03741ae659073159184cd3e/affine-2.3.0-py2.py3-none-any.whl
Collecting simplejson
[?25l  Downloading https://files.pythonhosted.org/packages/a8/04/377418ac1e530ce2a196b54c6552c018fdf1fe776718053efb1f216bffcd/simplejson-3.17.2-cp37-cp37m-manylinux2010_x86_64.whl (128kB)
     |████████████████████████████████| 133kB 50.8MB/s 
[?25hRequirement already satisfied: decorator<5,>=4.3 in /usr/local/lib/python3.7/dist-packages (from networkx->mapclassify>=2.4.2->pysal) (4.4.2)
Collecting amply>=0.1.2
  Downloading https://files.pythonhosted.org/packages/f3/c5/dfa09dd2595a2ab2ab4e6fa7bebef9565812722e1980d04b0edce5032066/amply-0.1.4-py3-none-any.whl
Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.7/dist-packages (from sympy->quantecon>=0.4.7->giddy>=2.3.3->pysal) (1.2.1)
Requirement already satisfied: llvmlite<0.35,>=0.34.0.dev0 in /usr/local/lib/python3.7/dist-packages (from numba>=0.38->quantecon>=0.4.7->giddy>=2.3.3->pysal) (0.34.0)
Collecting munch
  Downloading https://files.pythonhosted.org/packages/cc/ab/85d8da5c9a45e072301beb37ad7f833cd344e04c817d97e0cc75681d248f/munch-2.5.0-py2.py3-none-any.whl
Requirement already satisfied: docutils>=0.3 in /usr/local/lib/python3.7/dist-packages (from amply>=0.1.2->pulp->spopt>=0.1.1->pysal) (0.17.1)
Building wheels for collected packages: pysal, inequality, pointpats, mgwr, spglm, spint, spvcm, splot
  Building wheel for pysal (setup.py) ... [?25l[?25hdone
  Created wheel for pysal: filename=pysal-2.4.0-cp37-none-any.whl size=18960 sha256=18433de2e3dda444414e1e2c761aa3d3608a5916fc480052d2e9c69b623ffd5e
  Stored in directory: /root/.cache/pip/wheels/48/c0/2f/ec47118138a00fa2f359e4ad28fe87b393c0c03e2041986ae2
  Building wheel for inequality (setup.py) ... [?25l[?25hdone
  Created wheel for inequality: filename=inequality-1.0.0-cp37-none-any.whl size=11801 sha256=9a74350e046a5dbfd60135667a7f3fff3e7846484561c60980ee6838b01a8ce8
  Stored in directory: /root/.cache/pip/wheels/0e/cc/44/666696f89f7c7b13cc477bd9c1a9aef180be238550419007f2
  Building wheel for pointpats (setup.py) ... [?25l[?25hdone
  Created wheel for pointpats: filename=pointpats-2.2.0-cp37-none-any.whl size=60818 sha256=b290e5cb4083748ee99a8ae64ef7cabbf92aeab1c323326df1fe3b16384ba7f2
  Stored in directory: /root/.cache/pip/wheels/f9/28/f2/65c0993e68e25cef954ecd20abff91c6b11f0419f31b60e2ff
  Building wheel for mgwr (setup.py) ... [?25l[?25hdone
  Created wheel for mgwr: filename=mgwr-2.1.2-cp37-none-any.whl size=46374 sha256=dff46014724529ff0acf437425a37e0d2b46a473c28704ce0309f99d4d58919e
  Stored in directory: /root/.cache/pip/wheels/cf/11/e3/68692f1c637b23a8e2c6e200f6183c3502718b5f7de6dc630d
  Building wheel for spglm (setup.py) ... [?25l[?25hdone
  Created wheel for spglm: filename=spglm-1.0.8-cp37-none-any.whl size=38807 sha256=84dcf591ba90dde5fb8252175b8fa749ce659d637044bd405c5c49012190b90a
  Stored in directory: /root/.cache/pip/wheels/cd/1e/e3/c68bde79087fdc97cf65ec86e6d1d7ed2171e1baa1b0482363
  Building wheel for spint (setup.py) ... [?25l[?25hdone
  Created wheel for spint: filename=spint-1.0.7-cp37-none-any.whl size=31371 sha256=0aaa388fac403080a34d1b9f6b3c0f12bba24f791c59a1accb2ad944551e77ea
  Stored in directory: /root/.cache/pip/wheels/e1/f0/fe/754baccec76ae29184c37ea6112d881a883e044977bd963d08
  Building wheel for spvcm (setup.py) ... [?25l[?25hdone
  Created wheel for spvcm: filename=spvcm-0.3.0-cp37-none-any.whl size=5777202 sha256=17b9c7deab22820963c540135227bd12e4a9353a93f2b2f9fc8604aee8eb7461
  Stored in directory: /root/.cache/pip/wheels/8f/2e/c8/35a9abbb5c8508cfc597cdd2217c1b9b2cb82fc2f7fe18c11c
  Building wheel for splot (setup.py) ... [?25l[?25hdone
  Created wheel for splot: filename=splot-1.1.3-cp37-none-any.whl size=37867 sha256=e7be5e410c9b7745a31ce5684ac5d7868486b1ad547f2492086c0e84e6593de4
  Stored in directory: /root/.cache/pip/wheels/bd/4f/94/71caf6a544f341a1ef2788a7c96272e7e9dfebb04a1aecc24f
Successfully built pysal inequality pointpats mgwr spglm spint spvcm splot
ERROR: requests 2.23.0 has requirement urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1, but you'll have urllib3 1.26.5 which is incompatible.
ERROR: datascience 0.10.6 has requirement folium==0.2.1, but you'll have folium 0.8.3 which is incompatible.
ERROR: albumentations 0.1.12 has requirement imgaug<0.2.7,>=0.2.5, but you'll have imgaug 0.2.9 which is incompatible.
ERROR: pytest-cov 2.12.1 has requirement coverage>=5.2.1, but you'll have coverage 3.7.1 which is incompatible.
ERROR: pytest-cov 2.12.1 has requirement pytest>=4.6, but you'll have pytest 3.6.4 which is incompatible.
Installing collected packages: libpysal, access, esda, quantecon, mapclassify, giddy, inequality, opencv-contrib-python, pointpats, munch, click-plugins, cligj, fiona, pyproj, geopandas, segregation, rtree, spaghetti, spreg, spglm, mgwr, spint, spvcm, snuggs, affine, rasterio, pygeos, simplejson, rasterstats, tobler, splot, amply, pulp, spopt, urllib3, python-dateutil, pytest-cov, pysal
  Found existing installation: opencv-contrib-python 4.1.2.30
    Uninstalling opencv-contrib-python-4.1.2.30:
      Successfully uninstalled opencv-contrib-python-4.1.2.30
  Found existing installation: urllib3 1.24.3
    Uninstalling urllib3-1.24.3:
      Successfully uninstalled urllib3-1.24.3
  Found existing installation: python-dateutil 2.8.1
    Uninstalling python-dateutil-2.8.1:
      Successfully uninstalled python-dateutil-2.8.1
Successfully installed access-1.1.3 affine-2.3.0 amply-0.1.4 click-plugins-1.1.1 cligj-0.7.2 esda-2.3.6 fiona-1.8.20 geopandas-0.9.0 giddy-2.3.3 inequality-1.0.0 libpysal-4.4.0 mapclassify-2.4.2 mgwr-2.1.2 munch-2.5.0 opencv-contrib-python-4.5.2.52 pointpats-2.2.0 pulp-2.4 pygeos-0.10 pyproj-3.1.0 pysal-2.4.0 pytest-cov-2.12.1 python-dateutil-2.8.0 quantecon-0.5.0 rasterio-1.2.4 rasterstats-0.15.0 rtree-0.9.7 segregation-1.5.0 simplejson-3.17.2 snuggs-1.4.7 spaghetti-1.5.6 spglm-1.0.8 spint-1.0.7 splot-1.1.3 spopt-0.1.1 spreg-1.2.2 spvcm-0.3.0 tobler-0.7.0 urllib3-1.26.5




Collecting cartoframes
[?25l  Downloading https://files.pythonhosted.org/packages/a2/34/e20671c9b56b9d174078fa0556be3b7f848a98174cd5e81e12d8bcc63359/cartoframes-1.2.1-py2.py3-none-any.whl (244kB)
     |████████████████████████████████| 245kB 8.4MB/s 
[?25hRequirement already satisfied: geopandas<1.0,>=0.6.0 in /usr/local/lib/python3.7/dist-packages (from cartoframes) (0.9.0)
Requirement already satisfied: appdirs<2.0,>=1.4.3 in /usr/local/lib/python3.7/dist-packages (from cartoframes) (1.4.4)
Collecting semantic-version<3,>=2.8.0
  Downloading https://files.pythonhosted.org/packages/a5/15/00ef3b7888a10363b7c402350eda3acf395ff05bebae312d1296e528516a/semantic_version-2.8.5-py2.py3-none-any.whl
Collecting carto<2.0,>=1.11.2
  Downloading https://files.pythonhosted.org/packages/25/e1/d32f14e5827b818aec6509b03fc04867728440035a54878d08c108890a1e/carto-1.11.2-py3-none-any.whl
Requirement already satisfied: jinja2<3.0,>=2.10.1 in /usr/local/lib/python3.7/dist-packages (from cartoframes) (2.11.3)
Collecting unidecode<2.0,>=1.1.0
[?25l  Downloading https://files.pythonhosted.org/packages/9e/25/723487ca2a52ebcee88a34d7d1f5a4b80b793f179ee0f62d5371938dfa01/Unidecode-1.2.0-py2.py3-none-any.whl (241kB)
     |████████████████████████████████| 245kB 29.4MB/s 
[?25hRequirement already satisfied: pandas>=0.25.0 in /usr/local/lib/python3.7/dist-packages (from cartoframes) (1.1.5)
Requirement already satisfied: shapely>=1.6 in /usr/local/lib/python3.7/dist-packages (from geopandas<1.0,>=0.6.0->cartoframes) (1.7.1)
Requirement already satisfied: fiona>=1.8 in /usr/local/lib/python3.7/dist-packages (from geopandas<1.0,>=0.6.0->cartoframes) (1.8.20)
Requirement already satisfied: pyproj>=2.2.0 in /usr/local/lib/python3.7/dist-packages (from geopandas<1.0,>=0.6.0->cartoframes) (3.1.0)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python3.7/dist-packages (from carto<2.0,>=1.11.2->cartoframes) (2.23.0)
Collecting pyrestcli==0.6.11
  Downloading https://files.pythonhosted.org/packages/af/24/d3e7638e30f36396caa4ddc1e25d3f1b5ff76d91738166996e74a9cb4fc0/pyrestcli-0.6.11.tar.gz
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python3.7/dist-packages (from jinja2<3.0,>=2.10.1->cartoframes) (2.0.1)
Requirement already satisfied: pytz>=2017.2 in /usr/local/lib/python3.7/dist-packages (from pandas>=0.25.0->cartoframes) (2018.9)
Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.7/dist-packages (from pandas>=0.25.0->cartoframes) (2.8.0)
Requirement already satisfied: numpy>=1.15.4 in /usr/local/lib/python3.7/dist-packages (from pandas>=0.25.0->cartoframes) (1.19.5)
Requirement already satisfied: certifi in /usr/local/lib/python3.7/dist-packages (from fiona>=1.8->geopandas<1.0,>=0.6.0->cartoframes) (2020.12.5)
Requirement already satisfied: attrs>=17 in /usr/local/lib/python3.7/dist-packages (from fiona>=1.8->geopandas<1.0,>=0.6.0->cartoframes) (21.2.0)
Requirement already satisfied: click>=4.0 in /usr/local/lib/python3.7/dist-packages (from fiona>=1.8->geopandas<1.0,>=0.6.0->cartoframes) (7.1.2)
Requirement already satisfied: six>=1.7 in /usr/local/lib/python3.7/dist-packages (from fiona>=1.8->geopandas<1.0,>=0.6.0->cartoframes) (1.15.0)
Requirement already satisfied: munch in /usr/local/lib/python3.7/dist-packages (from fiona>=1.8->geopandas<1.0,>=0.6.0->cartoframes) (2.5.0)
Requirement already satisfied: cligj>=0.5 in /usr/local/lib/python3.7/dist-packages (from fiona>=1.8->geopandas<1.0,>=0.6.0->cartoframes) (0.7.2)
Requirement already satisfied: click-plugins>=1.0 in /usr/local/lib/python3.7/dist-packages (from fiona>=1.8->geopandas<1.0,>=0.6.0->cartoframes) (1.1.1)
Requirement already satisfied: setuptools in /usr/local/lib/python3.7/dist-packages (from fiona>=1.8->geopandas<1.0,>=0.6.0->cartoframes) (56.1.0)
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
[?25l  Downloading https://files.pythonhosted.org/packages/56/aa/4ef5aa67a9a62505db124a5cb5262332d1d4153462eb8fd89c9fa41e5d92/urllib3-1.25.11-py2.py3-none-any.whl (127kB)
     |████████████████████████████████| 133kB 38.9MB/s 
[?25hRequirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests>=2.7.0->carto<2.0,>=1.11.2->cartoframes) (2.10)
Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests>=2.7.0->carto<2.0,>=1.11.2->cartoframes) (3.0.4)
Requirement already satisfied: future>=0.15.2 in /usr/local/lib/python3.7/dist-packages (from pyrestcli==0.6.11->carto<2.0,>=1.11.2->cartoframes) (0.16.0)
Building wheels for collected packages: pyrestcli
  Building wheel for pyrestcli (setup.py) ... [?25l[?25hdone
  Created wheel for pyrestcli: filename=pyrestcli-0.6.11-cp37-none-any.whl size=8500 sha256=059b27982261a0e0d84cd8066b5807f22b318e373ff1c675d8894cba9279d514
  Stored in directory: /root/.cache/pip/wheels/2f/bb/11/396a62e2d1e718f2bfb02b66726240fbc8d98640bfc0cf1688
Successfully built pyrestcli
ERROR: pysal 2.4.0 has requirement urllib3>=1.26, but you'll have urllib3 1.25.11 which is incompatible.
ERROR: datascience 0.10.6 has requirement folium==0.2.1, but you'll have folium 0.8.3 which is incompatible.
Installing collected packages: semantic-version, pyrestcli, carto, unidecode, cartoframes, urllib3
  Found existing installation: urllib3 1.26.5
    Uninstalling urllib3-1.26.5:
      Successfully uninstalled urllib3-1.26.5
Successfully installed carto-1.11.2 cartoframes-1.2.1 pyrestcli-0.6.11 semantic-version-2.8.5 unidecode-1.2.0 urllib3-1.25.11
Collecting urllib3==1.22
[?25l  Downloading https://files.pythonhosted.org/packages/63/cb/6965947c13a94236f6d4b8223e21beb4d576dc72e8130bd7880f600839b8/urllib3-1.22-py2.py3-none-any.whl (132kB)
     |████████████████████████████████| 133kB 9.0MB/s 
ERROR: pysal 2.4.0 has requirement urllib3>=1.26, but you'll have urllib3 1.22 which is incompatible.
ERROR: datascience 0.10.6 has requirement folium==0.2.1, but you'll have folium 0.8.3 which is incompatible.
[?25hInstalling collected packages: urllib3
Successfully installed urllib3-1.22

Remember to restart the runtime to load correctly the installed libraries!

Now, we download from the web the dataset for this lab.

! wget https://geodacenter.github.io/data-and-lab//data/columbus.zip
! unzip columbus.zip
! wget https://geodacenter.github.io/data-and-lab//data/boston.zip
! unzip boston.zip
! wget https://geodacenter.github.io/data-and-lab//data/kingcounty.zip
! unzip kingcounty.zip
! wget https://github.com/vincnardelli/seai/raw/main/lab4/data/dataNUTS3.zip
! unzip dataNUTS3.zip

--2021-06-02 16:01:39--  https://geodacenter.github.io/data-and-lab//data/columbus.zip
Resolving geodacenter.github.io (geodacenter.github.io)... 185.199.108.153, 185.199.109.153, 185.199.110.153, ...
Connecting to geodacenter.github.io (geodacenter.github.io)|185.199.108.153|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 472230 (461K) [application/zip]
Saving to: ‘columbus.zip’

columbus.zip        100%[===================>] 461.16K  --.-KB/s    in 0.02s   

2021-06-02 16:01:39 (18.8 MB/s) - ‘columbus.zip’ saved [472230/472230]

Archive:  columbus.zip
   creating: columbus/
  inflating: columbus/.DS_Store      
   creating: __MACOSX/
   creating: __MACOSX/columbus/
  inflating: __MACOSX/columbus/._.DS_Store  
  inflating: columbus/columbus.csv   
  inflating: __MACOSX/columbus/._columbus.csv  
  inflating: columbus/columbus.dbf   
  inflating: __MACOSX/columbus/._columbus.dbf  
   creating: columbus/columbus.gdb/
  inflating: columbus/columbus.gdb/a00000001.gdbindexes  
  inflating: columbus/columbus.gdb/a00000001.gdbtable  
  inflating: columbus/columbus.gdb/a00000001.gdbtablx  
  inflating: columbus/columbus.gdb/a00000001.TablesByName.atx  
  inflating: columbus/columbus.gdb/a00000002.gdbtable  
  inflating: columbus/columbus.gdb/a00000002.gdbtablx  
  inflating: columbus/columbus.gdb/a00000003.gdbindexes  
  inflating: columbus/columbus.gdb/a00000003.gdbtable  
  inflating: columbus/columbus.gdb/a00000003.gdbtablx  
  inflating: columbus/columbus.gdb/a00000004.CatItemsByPhysicalName.atx  
  inflating: columbus/columbus.gdb/a00000004.CatItemsByType.atx  
  inflating: columbus/columbus.gdb/a00000004.FDO_UUID.atx  
  inflating: columbus/columbus.gdb/a00000004.gdbindexes  
  inflating: columbus/columbus.gdb/a00000004.gdbtable  
  inflating: columbus/columbus.gdb/a00000004.gdbtablx  
  inflating: columbus/columbus.gdb/a00000004.spx  
  inflating: columbus/columbus.gdb/a00000005.CatItemTypesByName.atx  
  inflating: columbus/columbus.gdb/a00000005.CatItemTypesByParentTypeID.atx  
  inflating: columbus/columbus.gdb/a00000005.CatItemTypesByUUID.atx  
  inflating: columbus/columbus.gdb/a00000005.gdbindexes  
  inflating: columbus/columbus.gdb/a00000005.gdbtable  
  inflating: columbus/columbus.gdb/a00000005.gdbtablx  
  inflating: columbus/columbus.gdb/a00000006.CatRelsByDestinationID.atx  
  inflating: columbus/columbus.gdb/a00000006.CatRelsByOriginID.atx  
  inflating: columbus/columbus.gdb/a00000006.CatRelsByType.atx  
  inflating: columbus/columbus.gdb/a00000006.FDO_UUID.atx  
  inflating: columbus/columbus.gdb/a00000006.gdbindexes  
  inflating: columbus/columbus.gdb/a00000006.gdbtable  
  inflating: columbus/columbus.gdb/a00000006.gdbtablx  
  inflating: columbus/columbus.gdb/a00000007.CatRelTypesByBackwardLabel.atx  
  inflating: columbus/columbus.gdb/a00000007.CatRelTypesByDestItemTypeID.atx  
  inflating: columbus/columbus.gdb/a00000007.CatRelTypesByForwardLabel.atx  
  inflating: columbus/columbus.gdb/a00000007.CatRelTypesByName.atx  
  inflating: columbus/columbus.gdb/a00000007.CatRelTypesByOriginItemTypeID.atx  
  inflating: columbus/columbus.gdb/a00000007.CatRelTypesByUUID.atx  
  inflating: columbus/columbus.gdb/a00000007.gdbindexes  
  inflating: columbus/columbus.gdb/a00000007.gdbtable  
  inflating: columbus/columbus.gdb/a00000007.gdbtablx  
  inflating: columbus/columbus.gdb/a00000009.gdbindexes  
  inflating: columbus/columbus.gdb/a00000009.gdbtable  
  inflating: columbus/columbus.gdb/a00000009.gdbtablx  
  inflating: columbus/columbus.gdb/a00000009.spx  
  inflating: columbus/columbus.gdb/gdb  
  inflating: columbus/columbus.gdb/timestamps  
  inflating: columbus/columbus.geojson  
  inflating: columbus/columbus.gpkg  
  inflating: columbus/columbus.html  
  inflating: __MACOSX/columbus/._columbus.html  
  inflating: columbus/columbus.kml   
  inflating: columbus/columbus.mid   
  inflating: columbus/columbus.mif   
  inflating: columbus/columbus.prj   
  inflating: columbus/columbus.shp   
  inflating: __MACOSX/columbus/._columbus.shp  
  inflating: columbus/columbus.shx   
  inflating: __MACOSX/columbus/._columbus.shx  
  inflating: columbus/columbus.sqlite  
  inflating: columbus/columbus.xlsx  
  inflating: __MACOSX/columbus/._columbus.xlsx  
  inflating: __MACOSX/._columbus     
--2021-06-02 16:01:39--  https://geodacenter.github.io/data-and-lab//data/boston.zip
Resolving geodacenter.github.io (geodacenter.github.io)... 185.199.108.153, 185.199.109.153, 185.199.110.153, ...
Connecting to geodacenter.github.io (geodacenter.github.io)|185.199.108.153|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 670702 (655K) [application/zip]
Saving to: ‘boston.zip’

boston.zip          100%[===================>] 654.98K  --.-KB/s    in 0.03s   

2021-06-02 16:01:39 (18.8 MB/s) - ‘boston.zip’ saved [670702/670702]

Archive:  boston.zip
   creating: boston/
  inflating: boston/.DS_Store        
   creating: __MACOSX/boston/
  inflating: __MACOSX/boston/._.DS_Store  
  inflating: boston/boston.csv       
  inflating: boston/boston.dbf       
  inflating: __MACOSX/boston/._boston.dbf  
  inflating: boston/boston.geojson   
  inflating: boston/boston.gml       
  inflating: boston/boston.gpkg      
  inflating: boston/boston.mid       
  inflating: boston/boston.mif       
  inflating: boston/boston.shp       
  inflating: __MACOSX/boston/._boston.shp  
  inflating: boston/boston.shx       
  inflating: __MACOSX/boston/._boston.shx  
  inflating: boston/boston.sqlite    
  inflating: boston/boston.txt       
  inflating: __MACOSX/boston/._boston.txt  
  inflating: boston/boston.xlsx      
  inflating: boston/boston.xsd       
  inflating: boston/boston_metadata.html  
  inflating: __MACOSX/boston/._boston_metadata.html  
  inflating: boston/style.css        
  inflating: __MACOSX/boston/._style.css  
  inflating: __MACOSX/._boston       
--2021-06-02 16:01:40--  https://geodacenter.github.io/data-and-lab//data/kingcounty.zip
Resolving geodacenter.github.io (geodacenter.github.io)... 185.199.108.153, 185.199.109.153, 185.199.110.153, ...
Connecting to geodacenter.github.io (geodacenter.github.io)|185.199.108.153|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 4061729 (3.9M) [application/zip]
Saving to: ‘kingcounty.zip’

kingcounty.zip      100%[===================>]   3.87M  --.-KB/s    in 0.1s    

2021-06-02 16:01:40 (34.9 MB/s) - ‘kingcounty.zip’ saved [4061729/4061729]

Archive:  kingcounty.zip
   creating: kingcounty/
  inflating: kingcounty/kc_house.dbf  
   creating: __MACOSX/kingcounty/
  inflating: __MACOSX/kingcounty/._kc_house.dbf  
  inflating: kingcounty/kc_house.prj  
  inflating: __MACOSX/kingcounty/._kc_house.prj  
  inflating: kingcounty/kc_house.shp  
  inflating: __MACOSX/kingcounty/._kc_house.shp  
  inflating: kingcounty/kc_house.shx  
  inflating: __MACOSX/kingcounty/._kc_house.shx  
  inflating: kingcounty/kc_house_data.csv  
  inflating: __MACOSX/kingcounty/._kc_house_data.csv  
  inflating: kingcounty/King_county_zip.dbf  
  inflating: __MACOSX/kingcounty/._King_county_zip.dbf  
  inflating: kingcounty/King_county_zip.geojson  
  inflating: __MACOSX/kingcounty/._King_county_zip.geojson  
  inflating: kingcounty/King_county_zip.prj  
  inflating: __MACOSX/kingcounty/._King_county_zip.prj  
  inflating: kingcounty/King_county_zip.shp  
  inflating: __MACOSX/kingcounty/._King_county_zip.shp  
  inflating: kingcounty/King_county_zip.shx  
  inflating: __MACOSX/kingcounty/._King_county_zip.shx  
  inflating: __MACOSX/._kingcounty   
--2021-06-02 16:01:40--  https://github.com/vincnardelli/seai2021/raw/main/lab2/data/dataNUTS3.zip
Resolving github.com (github.com)... 192.30.255.112
Connecting to github.com (github.com)|192.30.255.112|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://raw.githubusercontent.com/vincnardelli/seai2021/main/lab2/data/dataNUTS3.zip [following]
--2021-06-02 16:01:41--  https://raw.githubusercontent.com/vincnardelli/seai2021/main/lab2/data/dataNUTS3.zip
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133, 185.199.109.133, 185.199.110.133, ...
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 2507426 (2.4M) [application/zip]
Saving to: ‘dataNUTS3.zip’

dataNUTS3.zip       100%[===================>]   2.39M  --.-KB/s    in 0.08s   

2021-06-02 16:01:41 (30.4 MB/s) - ‘dataNUTS3.zip’ saved [2507426/2507426]

Archive:  dataNUTS3.zip
   creating: dataNUTS3/
  inflating: __MACOSX/._dataNUTS3    
  inflating: dataNUTS3/dataNUTS3.csv  
  inflating: dataNUTS3/NUTS_RG_10M_2013.dbf  
  inflating: __MACOSX/dataNUTS3/._NUTS_RG_10M_2013.dbf  
  inflating: dataNUTS3/NUTS_RG_10M_2013.sbx  
  inflating: __MACOSX/dataNUTS3/._NUTS_RG_10M_2013.sbx  
  inflating: dataNUTS3/NUTS_RG_10M_2013.shx  
  inflating: __MACOSX/dataNUTS3/._NUTS_RG_10M_2013.shx  
  inflating: dataNUTS3/NUTS_RG_10M_2013.shp.xml  
  inflating: __MACOSX/dataNUTS3/._NUTS_RG_10M_2013.shp.xml  
  inflating: dataNUTS3/NUTS_RG_10M_2013.shp  
  inflating: __MACOSX/dataNUTS3/._NUTS_RG_10M_2013.shp  
  inflating: dataNUTS3/NUTS_RG_10M_2013.sbn  
  inflating: __MACOSX/dataNUTS3/._NUTS_RG_10M_2013.sbn  
  inflating: dataNUTS3/NUTS_RG_10M_2013.prj  
  inflating: __MACOSX/dataNUTS3/._NUTS_RG_10M_2013.prj

Let’s start!

Loading and Plotting spatial data

import pandas as pd
import numpy as np

data = pd.read_csv("dataNUTS3/dataNUTS3.csv")
data.head()
NUTS_ID year alpine category pcgdp employ_1564 employ_1564_male employ_1564_female employ_5564 unempl educ_2564 fertility pop pop_dens ppopGE65 ppopLT15 research_exp state
0 AT113 1977 No Predominantly rural NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN AT
1 AT112 1977 No Predominantly rural NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN AT
2 AT111 1977 No Predominantly rural NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN AT
3 AT111 1978 No Predominantly rural NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN AT
4 AT112 1978 No Predominantly rural NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN AT 
data = data.loc[data['year']==2012]
data.head()
NUTS_ID year alpine category pcgdp employ_1564 employ_1564_male employ_1564_female employ_5564 unempl educ_2564 fertility pop pop_dens ppopGE65 ppopLT15 research_exp state
105 AT111 2012 No Predominantly rural 20600.0 70.4 75.9 64.8 38.7 4.6 13.9 1.30 37561.0 54.1 0.210884 0.126993 NaN AT
106 AT113 2012 No Predominantly rural 22900.0 70.4 75.9 64.8 38.7 4.6 13.9 1.30 97721.0 67.4 0.200735 0.128099 NaN AT
107 AT112 2012 No Predominantly rural 28500.0 70.4 75.9 64.8 38.7 4.6 13.9 1.30 150500.0 99.0 0.187960 0.136744 NaN AT
362 AT127 2012 No Predominantly urban 43200.0 72.6 77.2 68.1 42.9 4.6 17.2 1.49 320945.0 223.1 0.186381 0.147617 NaN AT
363 AT124 2012 No Predominantly rural 25300.0 72.6 77.2 68.1 42.9 4.6 17.2 1.49 219354.0 48.1 0.206958 0.135202 NaN AT 
data.describe()
year pcgdp employ_1564 employ_1564_male employ_1564_female employ_5564 unempl educ_2564 fertility pop pop_dens ppopGE65 ppopLT15 research_exp
count 1474.0 1240.000000 1398.000000 1398.000000 1398.000000 1398.000000 1397.000000 1398.000000 1358.000000 1.325000e+03 1372.000000 1323.000000 1323.000000 805.000000
mean 2012.0 26665.196617 65.765379 71.822532 59.724535 51.398641 8.743522 26.199928 1.593675 4.116850e+05 437.668659 0.183848 0.156838 1.371652
std 0.0 14948.025774 9.474021 7.892328 12.187490 11.899590 5.773396 8.612239 0.341548 5.986194e+05 1019.342152 0.042279 0.037709 1.011049
min 2012.0 1700.000000 28.300000 50.500000 6.500000 16.200000 2.700000 6.300000 1.060000 1.070700e+04 1.200000 0.026679 0.088176 0.070000
25% 2012.0 18675.000000 59.300000 66.900000 53.500000 40.700000 4.800000 19.300000 1.370000 1.490120e+05 64.175000 0.161477 0.134810 0.750000
50% 2012.0 25600.000000 67.050000 72.800000 62.600000 52.800000 7.400000 26.400000 1.460000 2.702010e+05 129.400000 0.187967 0.149737 1.190000
75% 2012.0 31900.000000 73.600000 78.200000 68.800000 61.850000 10.300000 32.200000 1.830000 4.993400e+05 323.125000 0.212095 0.170365 1.630000
max 2012.0 138156.096600 82.200000 87.800000 80.500000 79.100000 37.000000 51.200000 3.800000 1.362424e+07 21206.100000 0.294344 0.424929 11.050000 
data['category'].describe()

count             1127
unique               3
top       Intermediate
freq               450
Name: category, dtype: object

data['category'].value_counts()

Intermediate           450
Predominantly rural    416
Predominantly urban    261
Name: category, dtype: int64

data['state'].value_counts()

DE    396
UK    173
IT    110
FR    101
TR     81
PL     72
ES     59
EL     52
BE     44
RO     42
NL     40
AT     35
BG     28
CH     26
PT     25
SE     21
HR     21
HU     20
NO     19
FI     19
CZ     14
SI     12
DK     11
LT     10
SK      8
MK      8
IE      8
LV      6
EE      5
MT      2
IS      2
LU      1
CY      1
LI      1
ME      1
Name: state, dtype: int64

import seaborn as sns

data['pop_dens_log'] = np.log(data['pop_dens'])
data['pcgdp_log'] = np.log(data['pcgdp'])

sns.scatterplot(data=data, x="pop_dens_log", y="pcgdp_log")

<matplotlib.axes._subplots.AxesSubplot at 0x7fda44434cd0>

png

sns.scatterplot(data=data, x="pop_dens_log", y="pcgdp_log", hue="state", alpha=0.5, s=70, legend= False)

<matplotlib.axes._subplots.AxesSubplot at 0x7fda52649c50>

png

sns.lmplot(data=data, x="pop_dens_log", y="pcgdp_log", scatter_kws={'alpha':0.2})

<seaborn.axisgrid.FacetGrid at 0x7fda528d2950>

png

Areal data

GeoPandas is an open source project to make working with geospatial data in python easier. GeoPandas extends the datatypes used by pandas to allow spatial operations on geometric types. Geometric operations are performed by shapely. Geopandas further depends on fiona for file access and matplotlib for plotting.

https://geopandas.org/

The core data structure in GeoPandas is geopandas.GeoDataFrame, a subclass of pandas.DataFrame able to store geometry columns and perform spatial operations. Geometries are handled by geopandas.GeoSeries, a subclass of pandas.Series. Therefore, your GeoDataFrame is a combination of Series with your data (numerical, boolean, text etc.) and GeoSeries with geometries (points, polygons etc.). You can have as many columns with geometries as you wish, there’s no limit typical for desktop GIS software.

geopandas.png

It’s work with .geoJSON or PostGIS too!

import geopandas as gpd
gdf = gpd.read_file("dataNUTS3/NUTS_RG_10M_2013.shp")
gdf

/usr/local/lib/python3.7/dist-packages/geopandas/_compat.py:110: UserWarning: The Shapely GEOS version (3.8.0-CAPI-1.13.1 ) is incompatible with the GEOS version PyGEOS was compiled with (3.9.1-CAPI-1.14.2). Conversions between both will be slow.
  shapely_geos_version, geos_capi_version_string
NUTS_ID STAT_LEVL_ SHAPE_AREA SHAPE_LEN geometry
0 AT 0 10.026017 23.934459 POLYGON ((15.15733 48.99178, 15.16025 48.94169...
1 AT1 1 2.850834 11.222485 POLYGON ((15.15733 48.99178, 15.16025 48.94169...
2 AT11 2 0.473189 5.822330 POLYGON ((17.09271 48.09965, 17.06741 48.03144...
3 AT111 3 0.086556 1.232223 POLYGON ((16.64622 47.44660, 16.57575 47.40635...
4 AT112 3 0.212046 2.811666 POLYGON ((17.16080 48.00666, 17.09466 47.97087...
... ... ... ... ... ...
1946 ITI33 3 0.305710 2.834500 POLYGON ((13.66761 43.43056, 13.74297 43.29415...
1947 ITI34 3 0.134340 1.916392 POLYGON ((13.91552 42.89452, 13.71091 42.84333...
1948 ITI35 3 0.097793 1.801067 POLYGON ((13.84185 43.10317, 13.84945 43.06666...
1949 ITI4 2 1.879270 9.269680 POLYGON ((11.94108 42.68365, 12.03930 42.64881...
1950 ITI41 3 0.392265 3.510011 POLYGON ((11.94108 42.68365, 12.03930 42.64881...

1951 rows × 5 columns

gdf = gdf.merge(data, on="NUTS_ID", how="right")
gdf
NUTS_ID STAT_LEVL_ SHAPE_AREA SHAPE_LEN geometry year alpine category pcgdp employ_1564 employ_1564_male employ_1564_female employ_5564 unempl educ_2564 fertility pop pop_dens ppopGE65 ppopLT15 research_exp state pop_dens_log pcgdp_log
0 AT111 3 0.086556 1.232223 POLYGON ((16.64622 47.44660, 16.57575 47.40635... 2012 No Predominantly rural 20600.0 70.4 75.9 64.8 38.7 4.6 13.9 1.30 37561.0 54.1 0.210884 0.126993 NaN AT 3.990834 9.933046
1 AT113 3 0.174587 2.299135 POLYGON ((16.43376 47.35292, 16.48374 47.28760... 2012 No Predominantly rural 22900.0 70.4 75.9 64.8 38.7 4.6 13.9 1.30 97721.0 67.4 0.200735 0.128099 NaN AT 4.210645 10.038892
2 AT112 3 0.212046 2.811666 POLYGON ((17.16080 48.00666, 17.09466 47.97087... 2012 No Predominantly rural 28500.0 70.4 75.9 64.8 38.7 4.6 13.9 1.30 150500.0 99.0 0.187960 0.136744 NaN AT 4.595120 10.257659
3 AT127 3 0.181179 2.698516 POLYGON ((17.06674 48.11868, 17.03600 48.08430... 2012 No Predominantly urban 43200.0 72.6 77.2 68.1 42.9 4.6 17.2 1.49 320945.0 223.1 0.186381 0.147617 NaN AT 5.407620 10.673596
4 AT124 3 0.567886 3.891105 POLYGON ((15.54245 48.90770, 15.75363 48.85218... 2012 No Predominantly rural 25300.0 72.6 77.2 68.1 42.9 4.6 17.2 1.49 219354.0 48.1 0.206958 0.135202 NaN AT 3.873282 10.138560
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
1469 UKN01 3 0.016042 0.553346 POLYGON ((-5.91293 54.64804, -5.85534 54.63377... 2012 No Predominantly urban 49100.0 66.0 69.9 62.2 55.9 7.4 32.1 2.03 280717.0 2567.4 0.146104 0.176997 1.57 UK 7.850649 10.801614
1470 UKN04 3 0.450038 4.684459 POLYGON ((-5.97653 55.05660, -6.09788 54.98403... 2012 No Intermediate 19100.0 66.0 69.9 62.2 55.9 7.4 32.1 2.03 289557.0 90.1 0.141979 0.199432 1.57 UK 4.500920 9.857444
1471 UKN03 3 0.471958 6.393123 MULTIPOLYGON (((-5.99263 54.98926, -5.97045 54... 2012 No Intermediate 21500.0 66.0 69.9 62.2 55.9 7.4 32.1 2.03 441222.0 141.1 0.154099 0.196744 1.57 UK 4.949469 9.975808
1472 UKN05 3 0.902413 6.631712 POLYGON ((-6.50002 54.91871, -6.45709 54.82455... 2012 No Predominantly rural 20300.0 66.0 69.9 62.2 55.9 7.4 32.1 2.03 415192.0 66.6 0.133822 0.213335 1.57 UK 4.198705 9.918376
1473 UKN02 3 0.119908 2.793266 MULTIPOLYGON (((-5.77064 54.61195, -5.83296 54... 2012 No Predominantly urban 19200.0 66.0 69.9 62.2 55.9 7.4 32.1 2.03 392247.0 468.4 0.162120 0.188583 1.57 UK 6.149323 9.862666

1474 rows × 24 columns

gdf.plot(column='pop_dens', cmap='Reds')

<matplotlib.axes._subplots.AxesSubplot at 0x7fda2faca810>

png

EQUAL INTERVAL divides the data into equal size classes (e.g., 0-10, 10-20, 20-30, etc.) and works best on data that is generally spread across the entire range. CAUTION: Avoid equal interval if your data are skewed to one end or if you have one or two really large outlier values.

plt = gdf.plot(column='pop_dens', scheme='equal_interval', k=5, cmap='Reds', figsize=(10, 8), edgecolor='black',  linewidth=0.1)
plt.set_xlim(-11, 30)
plt.set_ylim(34, 70);

png

NATURAL BREAKS is a kind of “optimal” classification scheme that finds class breaks that will minimize within-class variance and maximize between-class differences.

plt = gdf.plot(column='pop_dens', scheme='natural_breaks', k=5, cmap='Reds', figsize=(10, 8), edgecolor='black',  linewidth=0.1)
plt.set_xlim(-11, 30)
plt.set_ylim(34, 70);

png

QUANTILES will create attractive maps that place an equal number of observations in each class.

plt = gdf.plot(column='pop_dens', scheme='Quantiles', k=5, cmap='Reds',  figsize=(10, 8), edgecolor='black',  linewidth=0.1)
plt.set_xlim(-11, 30)
plt.set_ylim(34, 70);

png

plt = gdf.plot(column='pop_dens', scheme='Quantiles', k=7, cmap='Blues',legend=True, figsize=(10, 8), edgecolor='black',  linewidth=0.1)
plt.set_xlim(-11, 30)
plt.set_ylim(34, 70);

png

Interactive plots with Carto Frames

CARTOframes is a Python package for integrating CARTO maps, analysis, and data services into data science workflows.

https://carto.com/developers/cartoframes

from cartoframes.viz import Layer, color_bins_style, formula_widget, category_widget, histogram_widget
Layer(
    gdf,
    color_bins_style('pop_dens', palette='CB_BLUES', bins=5, opacity=0.8)
)

Output hidden; open in https://colab.research.google.com to view.

from cartoframes.viz import Layer, color_bins_style, formula_widget, category_widget, histogram_widget
Layer(
    gdf,
    color_bins_style('pop_dens', palette='CB_BLUES', bins=5, opacity=0.8),
    default_widget=True,
    default_popup_hover=False,
    widgets=[
        formula_widget('pop', 'sum', 'Population'),
        formula_widget('pcgdp', 'sum', 'Per capita GDP'),
        category_widget('category', 'Category'),

        histogram_widget('unempl', 'Population density'),

        histogram_widget('pop_dens', 'Population density'),
        category_widget('category', 'Category'),
        category_widget('alpine', 'Alpine')
    ]
)


Output hidden; open in https://colab.research.google.com to view.

Point data

kc = gpd.read_file("kingcounty/kc_house.shp")
kc
id date price bedrooms bathrooms sqft_liv sqft_lot floors waterfront view condition grade sqft_above sqft_basmt yr_built yr_renov zipcode lat long sqft_liv15 sqft_lot15 geometry
0 7.129301e+09 20141013T000000 221900.0 3.0 1.0 1180.0 5650.0 1.0 0.0 0.0 3.0 7.0 1180.0 0.0 1955.0 0.0 98178.0 47.5112 -122.257 1340.0 5650.0 POINT (-122.25700 47.51120)
1 6.414100e+09 20141209T000000 538000.0 3.0 2.0 2570.0 7242.0 2.0 0.0 0.0 3.0 7.0 2170.0 400.0 1951.0 1991.0 98125.0 47.7210 -122.319 1690.0 7639.0 POINT (-122.31900 47.72100)
2 5.631500e+09 20150225T000000 180000.0 2.0 1.0 770.0 10000.0 1.0 0.0 0.0 3.0 6.0 770.0 0.0 1933.0 0.0 98028.0 47.7379 -122.233 2720.0 8062.0 POINT (-122.23300 47.73790)
3 2.487201e+09 20141209T000000 604000.0 4.0 3.0 1960.0 5000.0 1.0 0.0 0.0 5.0 7.0 1050.0 910.0 1965.0 0.0 98136.0 47.5208 -122.393 1360.0 5000.0 POINT (-122.39300 47.52080)
4 1.954401e+09 20150218T000000 510000.0 3.0 2.0 1680.0 8080.0 1.0 0.0 0.0 3.0 8.0 1680.0 0.0 1987.0 0.0 98074.0 47.6168 -122.045 1800.0 7503.0 POINT (-122.04500 47.61680)
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
21608 2.630000e+08 20140521T000000 360000.0 3.0 2.0 1530.0 1131.0 3.0 0.0 0.0 3.0 8.0 1530.0 0.0 2009.0 0.0 98103.0 47.6993 -122.346 1530.0 1509.0 POINT (-122.34600 47.69930)
21609 6.600060e+09 20150223T000000 400000.0 4.0 2.0 2310.0 5813.0 2.0 0.0 0.0 3.0 8.0 2310.0 0.0 2014.0 0.0 98146.0 47.5107 -122.362 1830.0 7200.0 POINT (-122.36200 47.51070)
21610 1.523300e+09 20140623T000000 402101.0 2.0 0.0 1020.0 1350.0 2.0 0.0 0.0 3.0 7.0 1020.0 0.0 2009.0 0.0 98144.0 47.5944 -122.299 1020.0 2007.0 POINT (-122.29900 47.59440)
21611 2.913101e+08 20150116T000000 400000.0 3.0 2.0 1600.0 2388.0 2.0 0.0 0.0 3.0 8.0 1600.0 0.0 2004.0 0.0 98027.0 47.5345 -122.069 1410.0 1287.0 POINT (-122.06900 47.53450)
21612 1.523300e+09 20141015T000000 325000.0 2.0 0.0 1020.0 1076.0 2.0 0.0 0.0 3.0 7.0 1020.0 0.0 2008.0 0.0 98144.0 47.5941 -122.299 1020.0 1357.0 POINT (-122.29900 47.59410)

21613 rows × 22 columns

Coordinate Reference Systems

A coordinate reference system (CRS) then defines how the two-dimensional, projected map in your GIS relates to real places on the earth. The decision of which map projection and CRS to use depends on the regional extent of the area you want to work in, on the analysis you want to do, and often on the availability of data.

https://datacarpentry.org/organization-geospatial/03-crs/

kc.crs

<Geographic 2D CRS: EPSG:4326>
Name: WGS 84
Axis Info [ellipsoidal]:
- Lat[north]: Geodetic latitude (degree)
- Lon[east]: Geodetic longitude (degree)
Area of Use:
- name: World.
- bounds: (-180.0, -90.0, 180.0, 90.0)
Datum: World Geodetic System 1984 ensemble
- Ellipsoid: WGS 84
- Prime Meridian: Greenwich

Let’s try to do the same starting from non-projected non-spatial dataset from a csv file!

kc = pd.read_csv("kingcounty/kc_house_data.csv")
kc.head()
id date price bedrooms bathrooms sqft_living sqft_lot floors waterfront view condition grade sqft_above sqft_basement yr_built yr_renovated zipcode lat long sqft_living15 sqft_lot15
0 7129300520 20141013T000000 221900.0 3 1.00 1180 5650 1.0 0 0 3 7 1180 0 1955 0 98178 47.5112 -122.257 1340 5650
1 6414100192 20141209T000000 538000.0 3 2.25 2570 7242 2.0 0 0 3 7 2170 400 1951 1991 98125 47.7210 -122.319 1690 7639
2 5631500400 20150225T000000 180000.0 2 1.00 770 10000 1.0 0 0 3 6 770 0 1933 0 98028 47.7379 -122.233 2720 8062
3 2487200875 20141209T000000 604000.0 4 3.00 1960 5000 1.0 0 0 5 7 1050 910 1965 0 98136 47.5208 -122.393 1360 5000
4 1954400510 20150218T000000 510000.0 3 2.00 1680 8080 1.0 0 0 3 8 1680 0 1987 0 98074 47.6168 -122.045 1800 7503 
kc = gpd.GeoDataFrame(kc, geometry=gpd.points_from_xy(kc.long, kc.lat, crs=4326))
kc.head()
id date price bedrooms bathrooms sqft_living sqft_lot floors waterfront view condition grade sqft_above sqft_basement yr_built yr_renovated zipcode lat long sqft_living15 sqft_lot15 geometry
0 7129300520 20141013T000000 221900.0 3 1.00 1180 5650 1.0 0 0 3 7 1180 0 1955 0 98178 47.5112 -122.257 1340 5650 POINT (-122.25700 47.51120)
1 6414100192 20141209T000000 538000.0 3 2.25 2570 7242 2.0 0 0 3 7 2170 400 1951 1991 98125 47.7210 -122.319 1690 7639 POINT (-122.31900 47.72100)
2 5631500400 20150225T000000 180000.0 2 1.00 770 10000 1.0 0 0 3 6 770 0 1933 0 98028 47.7379 -122.233 2720 8062 POINT (-122.23300 47.73790)
3 2487200875 20141209T000000 604000.0 4 3.00 1960 5000 1.0 0 0 5 7 1050 910 1965 0 98136 47.5208 -122.393 1360 5000 POINT (-122.39300 47.52080)
4 1954400510 20150218T000000 510000.0 3 2.00 1680 8080 1.0 0 0 3 8 1680 0 1987 0 98074 47.6168 -122.045 1800 7503 POINT (-122.04500 47.61680) 
kc.plot(column='price', cmap='Reds', legend=True, alpha=0.5, figsize=(10, 6));

png

kc.plot(column='price', cmap='coolwarm', legend=True, alpha=0.3, figsize=(10, 6), edgecolor='black',  linewidth=0.2);

png

kc.plot(column='price', scheme='Quantiles', k=5, cmap='Reds', legend=True, alpha=0.5, figsize=(10, 6));

png

from cartoframes.viz import Layer, color_bins_style, formula_widget, category_widget, histogram_widget, popup_element
Layer(
    kc,
    color_bins_style('price', palette='CB_BLUES', bins=5, opacity=0.8),
    default_widget=True,
    default_popup_hover=False,
    widgets=[
        formula_widget('id', 'count', 'Houses'),
        histogram_widget('sqft_lot', 'Sqft'),
        histogram_widget('bedrooms', 'Bedrooms')],
    popup_hover=[
        popup_element('price'),
        popup_element('bedrooms'),
        popup_element('bathrooms')
    ]
)

Computation of W matrix

https://pysal.org/packages/

Creation of a W matrix for regular grid data

To start, consider the case of a regular square lattice grid of dimension (e. g.) 5-by-5.

from libpysal.weights import lat2W

w = lat2W(5, 5)
print(w)

<libpysal.weights.weights.W object at 0x7fda233d9350>

w.n

25

w.neighbors

{0: [5, 1],
 1: [0, 6, 2],
 2: [1, 7, 3],
 3: [2, 8, 4],
 4: [3, 9],
 5: [0, 10, 6],
 6: [1, 5, 11, 7],
 7: [2, 6, 12, 8],
 8: [3, 7, 13, 9],
 9: [4, 8, 14],
 10: [5, 15, 11],
 11: [6, 10, 16, 12],
 12: [7, 11, 17, 13],
 13: [8, 12, 18, 14],
 14: [9, 13, 19],
 15: [10, 20, 16],
 16: [11, 15, 21, 17],
 17: [12, 16, 22, 18],
 18: [13, 17, 23, 19],
 19: [14, 18, 24],
 20: [15, 21],
 21: [16, 20, 22],
 22: [17, 21, 23],
 23: [18, 22, 24],
 24: [19, 23]}

w_card = pd.Series(w.cardinalities)
w_card.head()

0    2
1    3
2    3
3    3
4    2
dtype: int64

import seaborn as sns
sns.displot(w_card, bins=3);

png

Creation of a W matrix for irregular data

In the case of irregular data, it is possible to import a GAL (.gal).

#gal = libpysal.io.open(libpysal.examples.get_path('name_of_gal_file.gal'),'r')
#w = gal.read()
#gal.close()

Creation of W matrix from a GeoPandas GeoDataFrame

from libpysal.weights import Queen, Rook

columbus = gpd.read_file("columbus/columbus.shp")
columbus = columbus.to_crs(4326)
w_queen = Queen.from_dataframe(columbus)

w_queen.n

49

w_queen.neighbors

{0: [1, 2],
 1: [0, 2, 3],
 2: [0, 1, 3, 4],
 3: [1, 2, 4, 7],
 4: [2, 3, 5, 7, 8, 10, 14, 15],
 5: [8, 4],
 6: [11, 12, 13, 7],
 7: [3, 4, 6, 10, 11, 12],
 8: [4, 5, 9, 14, 19, 21, 24, 25],
 9: [16, 8, 19, 21],
 10: [4, 7, 11, 14, 15],
 11: [6, 7, 10, 12, 13, 15],
 12: [11, 13, 6, 7],
 13: [17, 18, 6, 11, 12, 15],
 14: [4, 8, 24, 10, 25, 15],
 15: [4, 10, 11, 13, 14, 17, 23, 24],
 16: [9, 19, 22],
 17: [18, 15, 13, 23],
 18: [17, 13, 23],
 19: [32, 34, 39, 8, 9, 16, 21, 22, 26, 31],
 20: [33, 29, 23],
 21: [19, 8, 9, 26, 27, 25],
 22: [16, 19, 31],
 23: [17, 18, 20, 24, 28, 29, 15],
 24: [8, 14, 15, 23, 25, 27, 28, 29],
 25: [21, 8, 24, 27, 28, 14],
 26: [32, 27, 19, 21],
 27: [32, 34, 36, 37, 21, 24, 25, 26, 28],
 28: [36, 37, 23, 24, 25, 27, 29],
 29: [20, 36, 23, 24, 28],
 30: [33, 35, 38],
 31: [40, 19, 22, 39],
 32: [26, 27, 34, 19],
 33: [41, 35, 20, 30],
 34: [32, 19, 37, 39, 27, 42, 43],
 35: [33, 38, 41, 45, 30],
 36: [37, 42, 27, 28, 29, 44],
 37: [34, 36, 43, 42, 27, 28],
 38: [35, 45, 30],
 39: [34, 19, 40, 46, 31],
 40: [31, 46, 39],
 41: [33, 35],
 42: [34, 36, 37, 43, 44, 47],
 43: [48, 34, 37, 42, 47],
 44: [48, 42, 36, 47],
 45: [35, 38],
 46: [40, 39],
 47: [48, 42, 43, 44],
 48: [43, 44, 47]}

w_queen[0]

{1: 1.0, 2: 1.0}

w_queen[0][1]

1.0

w_queen.mean_neighbors

4.816326530612245

w_queen.min_neighbors

2

w_queen.max_neighbors

10

w_queen.islands

[]

w_card = pd.Series(w_queen.cardinalities)
sns.displot(w_card, bins=10);

png

from splot.libpysal import plot_spatial_weights
plot_spatial_weights(w_queen, columbus);

/usr/local/lib/python3.7/dist-packages/splot/_viz_libpysal_mpl.py:115: UserWarning: Geometry is in a geographic CRS. Results from 'centroid' are likely incorrect. Use 'GeoSeries.to_crs()' to re-project geometries to a projected CRS before this operation.

  centroids_shp = gdf.centroid.values
/usr/local/lib/python3.7/dist-packages/splot/_viz_libpysal_mpl.py:154: UserWarning: Geometry is in a geographic CRS. Results from 'centroid' are likely incorrect. Use 'GeoSeries.to_crs()' to re-project geometries to a projected CRS before this operation.

  gdf.centroid.plot(ax=ax, **node_kws)

png

w_rook = Rook.from_dataframe(columbus)
plot_spatial_weights(w_rook, columbus);

/usr/local/lib/python3.7/dist-packages/splot/_viz_libpysal_mpl.py:115: UserWarning: Geometry is in a geographic CRS. Results from 'centroid' are likely incorrect. Use 'GeoSeries.to_crs()' to re-project geometries to a projected CRS before this operation.

  centroids_shp = gdf.centroid.values
/usr/local/lib/python3.7/dist-packages/splot/_viz_libpysal_mpl.py:154: UserWarning: Geometry is in a geographic CRS. Results from 'centroid' are likely incorrect. Use 'GeoSeries.to_crs()' to re-project geometries to a projected CRS before this operation.

  gdf.centroid.plot(ax=ax, **node_kws)

png

w_queen.pct_nonzero

9.82923781757601

w_rook.pct_nonzero

8.329862557267806

W Matrix for point data

boston = gpd.read_file("boston/boston.shp")
boston.head()
ID TOWN TOWNNO TRACT LON LAT x y MEDV CMEDV CRIM ZN INDUS CHAS NOX RM AGE DIS RAD TAX PTRATIO B LSTAT geometry
0 1.0 0.0 0.0 2011.0 -70.955 42.2550 338.73 4679.73 24.0 24.0 0.00632 18.0 2.31 0.0 0.538 6.575 65.2 4.0900 1.0 296.0 15.3 396.90 4.98 POINT (338.730 4679.730)
1 2.0 0.0 1.0 2021.0 -70.950 42.2875 339.23 4683.33 21.6 21.6 0.02731 0.0 7.07 0.0 0.469 6.421 78.9 4.9671 2.0 242.0 17.8 396.90 9.14 POINT (339.230 4683.330)
2 3.0 0.0 1.0 2022.0 -70.936 42.2830 340.37 4682.80 34.7 34.7 0.02729 0.0 7.07 0.0 0.469 7.185 61.1 4.9671 2.0 242.0 17.8 392.83 4.03 POINT (340.370 4682.800)
3 4.0 0.0 2.0 2031.0 -70.928 42.2930 341.05 4683.89 33.4 33.4 0.03237 0.0 2.18 0.0 0.458 6.998 45.8 6.0622 3.0 222.0 18.7 394.63 2.94 POINT (341.050 4683.890)
4 5.0 0.0 2.0 2032.0 -70.922 42.2980 341.56 4684.44 36.2 36.2 0.06905 0.0 2.18 0.0 0.458 7.147 54.2 6.0622 3.0 222.0 18.7 396.90 5.33 POINT (341.560 4684.440) 
boston.plot(column="MEDV", figsize=(10, 6), scheme='Quantiles', k=7, cmap='Blues',legend=True)

<matplotlib.axes._subplots.AxesSubplot at 0x7fda229a4cd0>

png

KNN

Creates nearest neighbor weights matrix based on k nearest neighbors.

from libpysal.weights import KNN

knn5 = KNN.from_dataframe(boston, k=5)
knn5

<libpysal.weights.distance.KNN at 0x7fda22ffb590>

knn5.n

506

knn5.mean_neighbors

5.0

knn5.neighbors

{0: [31, 29, 34, 32, 28],
 1: [26, 27, 28, 29, 25],
 2: [1, 3, 29, 6, 28],
 3: [4, 6, 2, 5, 1],
 4: [3, 5, 6, 8, 7],
 5: [4, 3, 8, 9, 6],
 6: [3, 4, 7, 2, 1],
 7: [8, 10, 11, 9, 12],
 8: [7, 10, 9, 11, 5],
 9: [10, 8, 7, 11, 61],
 10: [8, 9, 7, 11, 61],
 11: [10, 49, 48, 7, 8],
 12: [47, 46, 7, 48, 11],
 13: [26, 14, 25, 15, 27],
 14: [17, 24, 25, 13, 15],
 15: [14, 16, 17, 13, 24],
 16: [15, 45, 17, 14, 18],
 17: [14, 21, 22, 23, 24],
 18: [19, 21, 20, 37, 17],
 19: [20, 21, 18, 33, 22],
 20: [33, 19, 21, 22, 34],
 21: [22, 20, 17, 33, 19],
 22: [23, 33, 21, 32, 20],
 23: [32, 22, 24, 30, 33],
 24: [30, 23, 25, 32, 14],
 25: [27, 26, 24, 30, 28],
 26: [25, 27, 13, 1, 28],
 27: [28, 25, 26, 30, 31],
 28: [27, 29, 31, 25, 26],
 29: [28, 27, 1, 31, 26],
 30: [24, 31, 23, 32, 25],
 31: [30, 32, 28, 24, 27],
 32: [23, 30, 22, 24, 33],
 33: [20, 22, 34, 21, 32],
 34: [33, 32, 20, 22, 23],
 35: [36, 19, 18, 20, 33],
 36: [35, 37, 18, 19, 38],
 37: [18, 38, 36, 19, 35],
 38: [37, 89, 87, 36, 90],
 39: [42, 40, 86, 41, 84],
 40: [39, 84, 86, 42, 83],
 41: [42, 53, 40, 43, 39],
 42: [41, 39, 43, 40, 53],
 43: [44, 45, 51, 48, 46],
 44: [48, 47, 46, 43, 45],
 45: [46, 44, 47, 15, 16],
 46: [45, 47, 44, 12, 48],
 47: [48, 12, 46, 49, 44],
 48: [47, 49, 11, 44, 12],
 49: [48, 11, 47, 44, 50],
 50: [59, 51, 52, 49, 58],
 51: [52, 50, 53, 59, 43],
 52: [51, 50, 58, 53, 59],
 53: [51, 52, 43, 41, 50],
 54: [65, 41, 53, 55, 66],
 55: [56, 57, 52, 58, 53],
 56: [57, 55, 58, 52, 60],
 57: [56, 55, 58, 52, 60],
 58: [60, 59, 52, 50, 62],
 59: [50, 61, 60, 62, 58],
 60: [62, 59, 61, 58, 50],
 61: [62, 59, 60, 9, 10],
 62: [61, 60, 63, 59, 9],
 63: [62, 60, 61, 58, 59],
 64: [63, 62, 60, 61, 57],
 65: [66, 40, 82, 83, 41],
 66: [65, 67, 82, 68, 83],
 67: [68, 69, 82, 66, 81],
 68: [69, 67, 70, 66, 82],
 69: [68, 70, 67, 72, 71],
 70: [72, 69, 71, 73, 79],
 71: [74, 73, 79, 75, 78],
 72: [70, 73, 71, 193, 74],
 73: [74, 71, 72, 192, 75],
 74: [75, 73, 71, 78, 76],
 75: [74, 76, 78, 96, 71],
 76: [96, 75, 78, 97, 95],
 77: [95, 76, 78, 94, 93],
 78: [79, 75, 76, 77, 71],
 79: [78, 71, 80, 75, 77],
 80: [81, 83, 93, 85, 79],
 81: [80, 83, 82, 79, 93],
 82: [81, 83, 80, 67, 65],
 83: [81, 82, 80, 84, 85],
 84: [86, 85, 83, 93, 80],
 85: [84, 93, 86, 87, 80],
 86: [84, 85, 39, 87, 40],
 87: [88, 89, 92, 85, 38],
 88: [89, 91, 92, 87, 90],
 89: [88, 90, 91, 87, 38],
 90: [89, 91, 117, 88, 116],
 91: [88, 89, 90, 111, 116],
 92: [94, 88, 87, 91, 89],
 93: [85, 94, 77, 80, 92],
 94: [92, 93, 77, 95, 88],
 95: [96, 99, 97, 77, 76],
 96: [95, 97, 76, 99, 98],
 97: [96, 98, 99, 95, 76],
 98: [97, 175, 96, 99, 76],
 99: [97, 95, 96, 98, 175],
 100: [109, 104, 103, 108, 110],
 101: [133, 174, 173, 103, 172],
 102: [103, 174, 101, 133, 173],
 103: [102, 133, 174, 101, 104],
 104: [105, 103, 100, 106, 133],
 105: [106, 130, 129, 104, 132],
 106: [105, 107, 129, 130, 104],
 107: [108, 106, 112, 109, 105],
 108: [107, 109, 112, 106, 100],
 109: [100, 108, 110, 112, 107],
 110: [111, 109, 112, 100, 108],
 111: [113, 110, 112, 116, 115],
 112: [108, 113, 114, 107, 111],
 113: [111, 116, 115, 112, 114],
 114: [122, 115, 121, 113, 112],
 115: [114, 121, 118, 116, 113],
 116: [117, 113, 118, 115, 111],
 117: [118, 116, 119, 115, 90],
 118: [117, 116, 115, 119, 121],
 119: [117, 118, 495, 494, 116],
 120: [492, 125, 121, 124, 494],
 121: [115, 124, 120, 492, 122],
 122: [123, 114, 124, 121, 492],
 123: [122, 124, 126, 492, 114],
 124: [492, 125, 123, 122, 121],
 125: [492, 124, 120, 490, 126],
 126: [123, 124, 492, 125, 490],
 127: [128, 140, 139, 380, 137],
 128: [127, 137, 129, 139, 140],
 129: [137, 136, 130, 128, 105],
 130: [105, 132, 131, 129, 135],
 131: [132, 134, 171, 135, 130],
 132: [131, 134, 171, 130, 135],
 133: [101, 174, 103, 173, 134],
 134: [131, 132, 171, 135, 133],
 135: [169, 168, 131, 171, 132],
 136: [137, 129, 168, 157, 130],
 137: [136, 129, 128, 138, 139],
 138: [150, 139, 158, 137, 157],
 139: [140, 141, 138, 128, 137],
 140: [139, 127, 380, 141, 128],
 141: [148, 143, 139, 147, 149],
 142: [144, 143, 145, 147, 141],
 143: [144, 147, 142, 148, 141],
 144: [143, 145, 142, 147, 148],
 145: [144, 147, 152, 146, 143],
 146: [149, 151, 147, 150, 148],
 147: [148, 149, 143, 146, 145],
 148: [149, 147, 141, 146, 143],
 149: [148, 147, 146, 150, 151],
 150: [159, 149, 151, 146, 158],
 151: [146, 159, 150, 156, 149],
 152: [146, 151, 145, 153, 154],
 153: [154, 155, 152, 156, 151],
 154: [153, 155, 156, 151, 152],
 155: [154, 153, 156, 160, 152],
 156: [159, 160, 151, 154, 158],
 157: [161, 158, 136, 138, 168],
 158: [159, 150, 160, 157, 156],
 159: [150, 156, 158, 151, 160],
 160: [156, 158, 159, 151, 154],
 161: [157, 166, 162, 168, 158],
 162: [163, 161, 166, 165, 160],
 163: [162, 165, 166, 161, 164],
 164: [181, 218, 182, 163, 165],
 165: [166, 163, 162, 167, 161],
 166: [161, 168, 165, 169, 162],
 167: [170, 165, 169, 171, 172],
 168: [135, 169, 166, 136, 161],
 169: [135, 170, 168, 171, 166],
 170: [171, 169, 135, 167, 134],
 171: [170, 134, 131, 132, 169],
 172: [173, 101, 171, 174, 170],
 173: [174, 101, 172, 133, 102],
 174: [173, 101, 133, 102, 103],
 175: [178, 177, 102, 98, 174],
 176: [177, 187, 186, 175, 178],
 177: [176, 178, 186, 175, 179],
 178: [179, 177, 173, 172, 174],
 179: [180, 178, 186, 172, 177],
 180: [182, 179, 181, 183, 167],
 181: [182, 164, 180, 218, 183],
 182: [181, 183, 180, 217, 164],
 183: [184, 182, 217, 180, 181],
 184: [183, 185, 216, 217, 182],
 185: [184, 214, 215, 183, 186],
 186: [177, 179, 185, 176, 178],
 187: [176, 177, 190, 188, 186],
 188: [189, 205, 187, 213, 190],
 189: [188, 191, 192, 190, 187],
 190: [192, 187, 176, 98, 189],
 191: [192, 189, 190, 188, 193],
 192: [191, 190, 189, 73, 74],
 193: [194, 72, 73, 191, 192],
 194: [193, 72, 191, 73, 192],
 195: [203, 196, 205, 189, 188],
 196: [197, 198, 195, 199, 203],
 197: [196, 198, 199, 195, 203],
 198: [196, 197, 195, 194, 199],
 199: [200, 197, 202, 196, 203],
 200: [199, 251, 252, 250, 202],
 201: [238, 239, 250, 242, 204],
 202: [204, 203, 201, 251, 238],
 203: [202, 195, 204, 199, 206],
 204: [202, 201, 238, 239, 281],
 205: [188, 213, 206, 207, 214],
 206: [207, 208, 213, 209, 211],
 207: [206, 208, 209, 211, 210],
 208: [207, 209, 210, 206, 211],
 209: [211, 210, 208, 207, 212],
 210: [209, 235, 211, 208, 236],
 211: [209, 212, 210, 235, 234],
 212: [211, 215, 209, 210, 234],
 213: [214, 206, 212, 205, 207],
 214: [185, 215, 213, 212, 184],
 215: [212, 214, 216, 185, 184],
 216: [223, 221, 215, 184, 217],
 217: [219, 182, 183, 181, 184],
 218: [164, 181, 182, 363, 217],
 219: [217, 220, 216, 358, 218],
 220: [219, 224, 358, 221, 223],
 221: [223, 222, 216, 234, 220],
 222: [221, 234, 223, 235, 220],
 223: [221, 216, 222, 234, 220],
 224: [220, 358, 226, 225, 227],
 225: [360, 224, 359, 227, 268],
 226: [227, 224, 233, 231, 222],
 227: [226, 231, 228, 225, 224],
 228: [229, 227, 480, 225, 230],
 229: [228, 230, 231, 227, 480],
 230: [231, 278, 232, 229, 227],
 231: [230, 232, 227, 226, 233],
 232: [231, 279, 230, 233, 237],
 233: [226, 234, 232, 235, 237],
 234: [235, 222, 221, 211, 223],
 235: [234, 210, 211, 236, 209],
 236: [210, 237, 235, 208, 209],
 237: [236, 232, 233, 235, 279],
 238: [239, 201, 242, 281, 204],
 239: [238, 281, 201, 242, 204],
 240: [241, 282, 242, 281, 239],
 241: [240, 242, 243, 256, 244],
 242: [243, 238, 241, 239, 201],
 243: [242, 247, 244, 249, 241],
 244: [247, 245, 243, 246, 248],
 245: [246, 244, 247, 248, 255],
 246: [245, 248, 247, 253, 244],
 247: [244, 248, 246, 243, 245],
 248: [249, 247, 246, 245, 244],
 249: [248, 250, 247, 243, 251],
 250: [251, 249, 201, 242, 243],
 251: [250, 249, 201, 252, 248],
 252: [249, 251, 248, 253, 250],
 253: [246, 254, 248, 245, 252],
 254: [253, 255, 246, 245, 248],
 255: [245, 254, 246, 244, 253],
 256: [241, 244, 240, 243, 245],
 257: [258, 264, 421, 365, 259],
 258: [259, 257, 264, 362, 260],
 259: [258, 362, 260, 261, 264],
 260: [261, 362, 259, 262, 258],
 261: [260, 262, 263, 259, 264],
 262: [261, 260, 360, 263, 267],
 263: [266, 265, 261, 264, 422],
 264: [265, 421, 258, 257, 263],
 265: [264, 266, 422, 421, 263],
 266: [263, 265, 422, 479, 264],
 267: [262, 263, 266, 261, 360],
 268: [359, 360, 357, 361, 358],
 269: [483, 272, 484, 482, 273],
 270: [271, 272, 485, 484, 269],
 271: [270, 272, 292, 303, 485],
 272: [270, 269, 271, 292, 273],
 273: [269, 272, 291, 483, 274],
 274: [275, 276, 277, 278, 229],
 275: [274, 278, 277, 276, 230],
 276: [274, 275, 277, 278, 273],
 277: [275, 278, 276, 279, 274],
 278: [275, 279, 230, 277, 232],
 279: [232, 278, 230, 280, 277],
 280: [279, 281, 282, 277, 237],
 281: [282, 239, 280, 238, 240],
 282: [281, 280, 240, 277, 239],
 283: [240, 282, 276, 291, 277],
 284: [289, 285, 290, 295, 283],
 285: [284, 286, 289, 256, 287],
 286: [285, 287, 289, 284, 288],
 287: [288, 289, 298, 297, 300],
 288: [297, 296, 289, 287, 295],
 289: [288, 287, 284, 296, 297],
 290: [295, 296, 294, 291, 292],
 291: [273, 290, 292, 294, 276],
 292: [271, 294, 272, 270, 303],
 293: [297, 294, 296, 295, 303],
 294: [295, 292, 296, 293, 290],
 295: [296, 290, 294, 297, 293],
 296: [295, 297, 294, 293, 290],
 297: [296, 293, 288, 295, 294],
 298: [300, 299, 287, 288, 297],
 299: [300, 298, 301, 287, 288],
 300: [298, 299, 301, 288, 297],
 301: [302, 293, 300, 303, 297],
 302: [301, 329, 303, 330, 328],
 303: [271, 292, 293, 270, 294],
 304: [305, 306, 307, 464, 319],
 305: [487, 304, 306, 486, 465],
 306: [464, 465, 459, 458, 463],
 307: [309, 319, 308, 463, 306],
 308: [318, 319, 309, 313, 314],
 309: [308, 307, 312, 319, 313],
 310: [311, 312, 462, 452, 461],
 311: [310, 312, 313, 462, 452],
 312: [309, 313, 310, 308, 462],
 313: [308, 312, 309, 314, 318],
 314: [318, 313, 316, 308, 317],
 315: [316, 340, 314, 339, 323],
 316: [314, 315, 323, 322, 317],
 317: [318, 321, 314, 322, 316],
 318: [308, 319, 314, 317, 313],
 319: [308, 318, 307, 309, 313],
 320: [327, 321, 326, 317, 324],
 321: [322, 320, 324, 317, 323],
 322: [323, 321, 324, 316, 317],
 323: [322, 336, 321, 316, 324],
 324: [321, 336, 322, 326, 323],
 325: [326, 327, 333, 332, 330],
 326: [327, 325, 324, 320, 321],
 327: [320, 326, 321, 328, 324],
 328: [329, 330, 327, 326, 320],
 329: [328, 302, 330, 327, 320],
 330: [328, 331, 325, 329, 332],
 331: [332, 330, 325, 333, 328],
 332: [331, 333, 325, 330, 334],
 333: [334, 332, 325, 335, 326],
 334: [333, 335, 346, 332, 337],
 335: [336, 337, 334, 333, 324],
 336: [324, 323, 337, 338, 322],
 337: [338, 336, 335, 339, 323],
 338: [339, 337, 340, 336, 323],
 339: [340, 338, 323, 316, 315],
 340: [339, 338, 315, 316, 323],
 341: [342, 349, 343, 344, 348],
 342: [343, 341, 340, 344, 339],
 343: [342, 340, 338, 339, 344],
 344: [337, 348, 343, 338, 335],
 345: [346, 347, 334, 348, 333],
 346: [345, 347, 334, 333, 348],
 347: [345, 346, 348, 334, 333],
 348: [347, 344, 346, 334, 345],
 349: [350, 341, 342, 351, 348],
 350: [349, 351, 341, 352, 354],
 351: [352, 353, 354, 350, 355],
 352: [353, 351, 354, 355, 350],
 353: [354, 352, 355, 351, 350],
 354: [353, 355, 351, 352, 347],
 355: [354, 353, 352, 351, 347],
 356: [363, 361, 163, 162, 218],
 357: [358, 268, 359, 361, 363],
 358: [357, 359, 268, 220, 224],
 359: [268, 360, 357, 361, 358],
 360: [359, 268, 262, 361, 260],
 361: [362, 363, 268, 359, 357],
 362: [259, 260, 361, 258, 261],
 363: [356, 361, 357, 362, 218],
 364: [365, 366, 153, 367, 257],
 365: [364, 366, 257, 367, 421],
 366: [367, 411, 365, 417, 412],
 367: [411, 366, 410, 368, 409],
 368: [369, 410, 367, 407, 409],
 369: [368, 367, 410, 370, 407],
 370: [371, 372, 407, 369, 368],
 371: [370, 372, 406, 374, 369],
 372: [406, 371, 374, 370, 373],
 373: [374, 406, 375, 372, 376],
 374: [373, 406, 372, 375, 371],
 375: [378, 377, 376, 373, 379],
 376: [375, 377, 378, 379, 373],
 377: [379, 378, 375, 376, 381],
 378: [377, 375, 379, 381, 376],
 379: [381, 377, 378, 375, 376],
 380: [381, 140, 379, 378, 377],
 381: [379, 377, 380, 378, 375],
 382: [383, 387, 388, 389, 489],
 383: [382, 388, 387, 389, 386],
 384: [386, 385, 387, 383, 388],
 385: [386, 384, 392, 387, 388],
 386: [384, 385, 387, 392, 388],
 387: [388, 386, 383, 384, 385],
 388: [387, 383, 389, 386, 384],
 389: [388, 383, 382, 387, 390],
 390: [389, 391, 488, 388, 383],
 391: [390, 503, 500, 491, 499],
 392: [385, 386, 387, 384, 388],
 393: [394, 395, 397, 396, 398],
 394: [395, 393, 396, 397, 402],
 395: [394, 396, 397, 393, 402],
 396: [395, 402, 397, 401, 394],
 397: [396, 395, 401, 394, 402],
 398: [397, 405, 401, 393, 399],
 399: [400, 405, 401, 404, 402],
 400: [404, 401, 399, 402, 403],
 401: [400, 402, 399, 404, 396],
 402: [403, 401, 396, 400, 404],
 403: [404, 402, 400, 401, 439],
 404: [400, 403, 399, 402, 401],
 405: [399, 401, 400, 404, 397],
 406: [372, 374, 373, 371, 375],
 407: [410, 409, 408, 368, 370],
 408: [409, 410, 412, 413, 411],
 409: [408, 410, 411, 412, 407],
 410: [409, 407, 408, 368, 411],
 411: [412, 409, 367, 410, 408],
 412: [413, 411, 417, 408, 409],
 413: [412, 417, 408, 414, 411],
 414: [413, 415, 416, 438, 412],
 415: [438, 416, 437, 429, 414],
 416: [415, 438, 428, 418, 417],
 417: [418, 412, 413, 411, 416],
 418: [417, 427, 416, 425, 419],
 419: [420, 425, 418, 427, 421],
 420: [419, 421, 425, 422, 423],
 421: [420, 265, 264, 422, 419],
 422: [479, 265, 423, 420, 266],
 423: [424, 425, 479, 422, 420],
 424: [478, 426, 423, 430, 476],
 425: [427, 419, 426, 423, 420],
 426: [478, 427, 428, 477, 424],
 427: [428, 425, 426, 418, 477],
 428: [477, 427, 426, 429, 478],
 429: [437, 477, 428, 436, 438],
 430: [476, 478, 475, 431, 477],
 431: [475, 432, 430, 435, 436],
 432: [475, 431, 474, 434, 430],
 433: [450, 434, 455, 432, 474],
 434: [435, 433, 450, 449, 432],
 435: [436, 434, 449, 445, 431],
 436: [445, 437, 435, 429, 431],
 437: [438, 436, 444, 429, 445],
 438: [415, 437, 444, 429, 416],
 439: [444, 440, 443, 403, 442],
 440: [442, 439, 443, 403, 444],
 441: [447, 446, 442, 443, 448],
 442: [443, 440, 441, 446, 439],
 443: [446, 442, 444, 445, 439],
 444: [437, 439, 445, 438, 443],
 445: [436, 437, 444, 446, 443],
 446: [443, 447, 445, 449, 441],
 447: [441, 448, 446, 449, 443],
 448: [451, 447, 449, 450, 446],
 449: [435, 448, 434, 446, 450],
 450: [433, 434, 451, 449, 454],
 451: [448, 453, 450, 454, 452],
 452: [461, 453, 451, 448, 447],
 453: [451, 454, 461, 460, 452],
 454: [453, 450, 455, 451, 460],
 455: [433, 454, 456, 450, 474],
 456: [457, 455, 458, 466, 454],
 457: [466, 458, 456, 465, 459],
 458: [457, 459, 460, 464, 456],
 459: [458, 460, 464, 463, 457],
 460: [459, 454, 453, 458, 461],
 461: [453, 452, 462, 460, 451],
 462: [463, 461, 459, 460, 452],
 463: [462, 459, 461, 460, 464],
 464: [306, 459, 458, 465, 463],
 465: [466, 457, 458, 464, 487],
 466: [457, 465, 456, 458, 464],
 467: [469, 473, 474, 468, 455],
 468: [470, 469, 487, 471, 467],
 469: [468, 470, 467, 471, 472],
 470: [468, 469, 471, 487, 472],
 471: [472, 482, 470, 468, 481],
 472: [471, 481, 470, 482, 469],
 473: [467, 479, 472, 469, 267],
 474: [432, 433, 476, 475, 434],
 475: [431, 432, 430, 435, 476],
 476: [430, 478, 475, 432, 431],
 477: [428, 478, 426, 429, 430],
 478: [477, 426, 430, 424, 476],
 479: [422, 423, 266, 265, 420],
 480: [481, 482, 228, 472, 483],
 481: [480, 482, 472, 471, 483],
 482: [471, 481, 483, 472, 480],
 483: [269, 482, 481, 480, 471],
 484: [486, 485, 269, 487, 270],
 485: [484, 486, 270, 305, 487],
 486: [484, 487, 485, 305, 468],
 487: [486, 468, 465, 470, 305],
 488: [489, 491, 490, 389, 382],
 489: [488, 490, 382, 383, 491],
 490: [489, 488, 125, 491, 126],
 491: [488, 490, 489, 493, 125],
 492: [124, 125, 120, 123, 126],
 493: [494, 498, 491, 495, 120],
 494: [493, 495, 498, 496, 120],
 495: [494, 496, 119, 493, 498],
 496: [497, 495, 498, 494, 499],
 497: [496, 499, 498, 500, 495],
 498: [499, 493, 494, 496, 497],
 499: [498, 500, 497, 496, 391],
 500: [499, 501, 391, 502, 497],
 501: [502, 504, 500, 503, 505],
 502: [503, 504, 501, 505, 500],
 503: [502, 504, 501, 391, 505],
 504: [502, 505, 503, 501, 500],
 505: [504, 502, 503, 501, 500]}

plot_spatial_weights(knn5, boston);

png

Kernel Weights

Spatial weights based on kernel functions.

For fixed bandwidth, \(h_{i}=\max (d k n n) \forall i\) where $d k n n$ is a vector of k-nearest neighbor distances (the distance to the kth nearest neighbor for each observation).

For adaptive bandwidths, \(h_{i}=d k n n_{i}\)

Kernel function defined as follows with: \(z_{i, j}=d_{i, j} / h_{i}\) triangular \(K(z)=(1-|z|) i f|z| \leq 1\) uniform \(K(z)=1 / 2 \text { if }|z| \leq 1\) quadratic \(K(z)=(3 / 4)\left(1-z^{2}\right) i f|z| \leq 1\) quartic \(K(z)=(15 / 16)\left(1-z^{2}\right)^{2} i f|z| \leq 1\) gaussian \(K(z)=(2 \pi)^{(-1 / 2)} \exp \left(-z^{2} / 2\right)\)

https://pysal.org/libpysal/generated/libpysal.weights.Kernel.html

from libpysal.weights import distance
w_kernel = distance.Kernel.from_dataframe(boston)

w_kernel.function

'triangular'

w_kernel.bandwidth[0:10]

array([[5.27056976],
       [5.27056976],
       [5.27056976],
       [5.27056976],
       [5.27056976],
       [5.27056976],
       [5.27056976],
       [5.27056976],
       [5.27056976],
       [5.27056976]])

w_kernel.weights[0]

[0.163551559050497,
 0.19181618325114325,
 0.09473389104628172,
 0.13092225033037153,
 0.07496742804829448,
 0.08191080022181252,
 0.1261552782976385,
 0.19097921738634083,
 0.1720602864115155,
 0.3010077516629547,
 0.10995087934229786,
 0.31958894493655776,
 0.2733728853634749,
 0.36773125208680824,
 0.27811466105273197,
 1.0,
 0.35814762302133973,
 0.4164829256867967,
 0.497978319440454,
 0.5124309808630476,
 0.5261957306180548,
 0.4742346749992036,
 0.2285687077374301,
 0.29823222830546314,
 0.17614003438822645,
 0.309349123035617,
 0.4617704592448053,
 0.2733134374315873,
 0.42287221770873373,
 0.27605558105763905,
 0.396646687116729,
 0.38536818952869667,
 0.20741404543113307,
 0.11238701394849415,
 0.37411750639295605,
 0.4722016578165268,
 0.32215713806517166,
 0.41625162522416426,
 0.31040541399739163,
 0.33961806947625883,
 0.06727845568338509,
 0.09626616719351899]

plot_spatial_weights(w_kernel, boston);

png

w_kernel_gaussian = distance.Kernel.from_dataframe(boston, function="gaussian", fixed=False, k=20)

w_kernel_gaussian.weights[0]

[0.3989422804014327,
 0.31705773159802025,
 0.3127930500228602,
 0.3082482854503523,
 0.30064563932915683,
 0.29998736630684864,
 0.2965929724745923,
 0.2837127108292177,
 0.28156780010707066,
 0.2814900153816355,
 0.27486510267232905,
 0.2710281274575302,
 0.2671846662070748,
 0.2649967458134385,
 0.2617059886370584,
 0.25532273799876404,
 0.2492897977719749,
 0.24840149007380918,
 0.24522374826057816,
 0.2448581503743604,
 0.2419707487162134]

plot_spatial_weights(w_kernel_gaussian, boston);

png

Distance

w_distance15 = distance.DistanceBand.from_dataframe(boston, 1.5, binary=True)

/usr/local/lib/python3.7/dist-packages/libpysal/weights/weights.py:172: UserWarning: The weights matrix is not fully connected: 
 There are 64 disconnected components.
 There are 49 islands with ids: 0, 40, 54, 55, 64, 65, 66, 67, 195, 196, 197, 198, 199, 200, 202, 203, 204, 252, 253, 254, 255, 256, 273, 283, 284, 285, 286, 287, 289, 291, 299, 301, 302, 303, 330, 335, 341, 342, 343, 344, 347, 348, 349, 350, 351, 352, 353, 354, 355.
  warnings.warn(message)

w_distance15.pct_nonzero

2.596509865800122

w_distance15.islands

[0,
 40,
 54,
 55,
 64,
 65,
 66,
 67,
 195,
 196,
 197,
 198,
 199,
 200,
 202,
 203,
 204,
 252,
 253,
 254,
 255,
 256,
 273,
 283,
 284,
 285,
 286,
 287,
 289,
 291,
 299,
 301,
 302,
 303,
 330,
 335,
 341,
 342,
 343,
 344,
 347,
 348,
 349,
 350,
 351,
 352,
 353,
 354,
 355]

plot_spatial_weights(w_distance15, boston);

png

w_distance5 = distance.DistanceBand.from_dataframe(boston, 5, binary=True)

w_distance5.pct_nonzero

20.07139620990798

plot_spatial_weights(w_distance5, boston);

png

Kriging

https://geostat-framework.readthedocs.io/projects/pykrige/en/stable/contents.html#

! pip3 install pykrige

Collecting pykrige
[?25l  Downloading https://files.pythonhosted.org/packages/b5/bc/f4123cd138a515f325fe1c02f4ccd508503aea1d06f10cec379a0c27e725/PyKrige-1.6.0-cp37-cp37m-manylinux2010_x86_64.whl (957kB)
     |████████████████████████████████| 962kB 7.9MB/s 
[?25hRequirement already satisfied: scipy>=1.1.0 in /usr/local/lib/python3.7/dist-packages (from pykrige) (1.4.1)
Requirement already satisfied: numpy>=1.14.5 in /usr/local/lib/python3.7/dist-packages (from pykrige) (1.19.5)
Installing collected packages: pykrige
Successfully installed pykrige-1.6.0

import numpy as np
import pykrige.kriging_tools as kt
from pykrige.ok import OrdinaryKriging
import matplotlib.pyplot as plt


data = np.array(
    [
        [0.3, 1.2, 0.47],
        [1.9, 0.6, 0.56],
        [1.1, 3.2, 0.74],
        [3.3, 4.4, 1.47],
        [4.7, 3.8, 1.74],
    ]
)

gridx = np.arange(0.0, 5.5, 0.5)
gridy = np.arange(0.0, 5.5, 0.5)

OK = OrdinaryKriging(
    data[:, 0],
    data[:, 1],
    data[:, 2],
    variogram_model="linear",
    verbose=True,
    enable_plotting=True,
    enable_statistics = True
)


Plotting Enabled

Adjusting data for anisotropy...
Initializing variogram model...
Coordinates type: 'euclidean' 

Using 'linear' Variogram Model
Slope: 0.11914486053372102
Nugget: 1.562799151461731e-10

png

Calculating statistics on variogram model fit...
Q1 = 0.7185997466464226
Q2 = 0.5261655056850497
cR = 0.22836851705603034 

z, ss = OK.execute("grid", gridx, gridy)

Executing Ordinary Kriging...

OK.display_variogram_model()

png

OK.get_epsilon_residuals()

array([0.11939362, 0.31524397, 1.13335687, 0.99419705])

OK.plot_epsilon_residuals()

png

OK.print_statistics()

Q1 = 0.8540638346446238
Q2 = 0.7955197182710158
cR = 0.4108295341866522

plt.imshow(z)

<matplotlib.image.AxesImage at 0x7fda222f59d0>

png

ss

masked_array(
  data=[[0.25923484621514353, 0.21697173215927953, 0.18562005009310606,
         0.15070036160851258, 0.1409807977110183, 0.19349623439414743,
         0.27091057749162145, 0.34859396413807986, 0.4218274091144686,
         0.4914972861602811, 0.5599388590592126],
        [0.1685790750642299, 0.13433839128069075, 0.12282338387603808,
         0.07928152497428621, 0.033412197372541405, 0.13348943280667194,
         0.22452086202401816, 0.3032191203296969, 0.37293023486324334,
         0.43771179023565937, 0.5016155100223009],
        [0.08513321025062208, 0.057344001806422594, 0.0988201151086116,
         0.09207956894947228, 0.0831828752132979, 0.14137090557413895,
         0.21305277711097992, 0.2779203059340609, 0.3357999549087324,
         0.3905481098685688, 0.44686986881283053],
        [0.09191464967628578, 0.07010101583643995, 0.11140934135535302,
         0.12986229606731423, 0.14347812569342044, 0.17561920857304442,
         0.21951485895469935, 0.2632407706365101, 0.3039890173487808,
         0.34502695685402585, 0.3918395827580563],
        [0.14948214785440378, 0.1217475985806349, 0.12854881481594435,
         0.14615953270233276, 0.16830020601219053, 0.19453240208520173,
         0.22194674394560182, 0.24688656697346245, 0.26945761171205823,
         0.295046175074278, 0.3320008321431087],
        [0.18356850506947925, 0.13375394095649235, 0.11179043555732382,
         0.1292000992277427, 0.1638315723909173, 0.1922874069134982,
         0.21092004612934748, 0.22122630209830302, 0.22624840776159327,
         0.23505804793713853, 0.26320910634511757],
        [0.20473710925155325, 0.12341468403522206, 0.047664586806003544,
         0.0878067281803101, 0.14560488298636257, 0.1756929458350838,
         0.18605280230202975, 0.18464159922712753, 0.1733635576216061,
         0.161427003572866, 0.18313405454162326],
        [0.23880816509066907, 0.14857123026131117, 0.07168774750225619,
         0.09555229371525809, 0.13971346731080803, 0.15425752100036777,
         0.14803555190518472, 0.13852577609948494, 0.11991632336379174,
         0.07613074414793403, 0.09809025883238591],
        [0.29627771686047677, 0.21928741671751967, 0.16671941780353483,
         0.154640117045358, 0.1555778354210307, 0.13660241671581147,
         0.09487459548352549, 0.08099406302375851, 0.0913473847441243,
         0.05590106420059135, 0.08496678734127737],
        [0.36864012548353403, 0.301222610214792, 0.2514557631550581,
         0.21967591573605158, 0.1908443620848134, 0.1446775191834751,
         0.06874055530188111, 0.04813430003152433, 0.10761668023856749,
         0.12913542357871338, 0.16666691988014123],
        [0.4479163263002224, 0.38482474342694295, 0.3333107579320962,
         0.29083217969923514, 0.2490868172716495, 0.20002319469798968,
         0.1504203103061949, 0.1406596133305867, 0.1738847320860175,
         0.2104606566988974, 0.25584260269013936]],
  mask=[[False, False, False, False, False, False, False, False, False,
         False, False],
        [False, False, False, False, False, False, False, False, False,
         False, False],
        [False, False, False, False, False, False, False, False, False,
         False, False],
        [False, False, False, False, False, False, False, False, False,
         False, False],
        [False, False, False, False, False, False, False, False, False,
         False, False],
        [False, False, False, False, False, False, False, False, False,
         False, False],
        [False, False, False, False, False, False, False, False, False,
         False, False],
        [False, False, False, False, False, False, False, False, False,
         False, False],
        [False, False, False, False, False, False, False, False, False,
         False, False],
        [False, False, False, False, False, False, False, False, False,
         False, False],
        [False, False, False, False, False, False, False, False, False,
         False, False]],
  fill_value=1e+20)

https://geostat-framework.readthedocs.io/projects/pykrige/en/stable/variogram_models.html