{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Categorical Variables" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In the last section we talked about scaling for continuous features, which all the features in the cancer dataset were.\n", "In the lending club data, we also saw another type of feature, categorical or discrete features. Categorical or discrete features are those can take one of several distinct values that are usually not numerical, and often not even ordered. In the lending club data, examples were the grade, which could be a letter from 'A' to 'G', or home ownership, which could be 'RENT', 'MORTGAGE', 'OWN' or 'ANY'.\n" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import pandas as pd" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "C:\\Users\\t3kci\\anaconda3\\lib\\site-packages\\IPython\\core\\interactiveshell.py:3063: DtypeWarning: Columns (123,124,125,128,129,130,133,139,140,141) have mixed types.Specify dtype option on import or set low_memory=False.\n", " interactivity=interactivity, compiler=compiler, result=result)\n" ] } ], "source": [ "loans = pd.read_csv(\"C:/Users/t3kci/Downloads/loan.csv/loan.csv\", nrows=100000)" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array(['C', 'D', 'B', 'A', 'E', 'F', 'G'], dtype=object)" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "loans.grade.unique()" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array(['RENT', 'MORTGAGE', 'OWN', 'ANY'], dtype=object)" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "loans.home_ownership.unique()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Most machine learning algorithms require that we preprocess these features into some numeric encoding before we can use them. For our old friend ``KNeighborsClassifier``, we could in theory define a distance metric on these categorical features (which is certainly possible and many options exist), though the simpler and more common approach is to transform the data so we can use the standard euclidean distance on all the features.\n", "Let's take a small subset of the data to investigate the possible preprocessing methods:" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
loan_amnthome_ownershipgradeloan_status
08000MORTGAGEACharged Off
16000MORTGAGECCharged Off
210000MORTGAGEACharged Off
310000RENTECharged Off
435000MORTGAGECCharged Off
54800MORTGAGECCharged Off
635000RENTCCharged Off
715000OWNBCharged Off
816000RENTBCharged Off
925000MORTGAGEACharged Off
1030000MORTGAGEDFully Paid
1140000MORTGAGECFully Paid
1220000MORTGAGEAFully Paid
134500RENTBFully Paid
148425MORTGAGEEFully Paid
1520000RENTDFully Paid
166600RENTBFully Paid
172500RENTCFully Paid
184000MORTGAGEDFully Paid
192700OWNAFully Paid
\n", "
" ], "text/plain": [ " loan_amnt home_ownership grade loan_status\n", "0 8000 MORTGAGE A Charged Off\n", "1 6000 MORTGAGE C Charged Off\n", "2 10000 MORTGAGE A Charged Off\n", "3 10000 RENT E Charged Off\n", "4 35000 MORTGAGE C Charged Off\n", "5 4800 MORTGAGE C Charged Off\n", "6 35000 RENT C Charged Off\n", "7 15000 OWN B Charged Off\n", "8 16000 RENT B Charged Off\n", "9 25000 MORTGAGE A Charged Off\n", "10 30000 MORTGAGE D Fully Paid\n", "11 40000 MORTGAGE C Fully Paid\n", "12 20000 MORTGAGE A Fully Paid\n", "13 4500 RENT B Fully Paid\n", "14 8425 MORTGAGE E Fully Paid\n", "15 20000 RENT D Fully Paid\n", "16 6600 RENT B Fully Paid\n", "17 2500 RENT C Fully Paid\n", "18 4000 MORTGAGE D Fully Paid\n", "19 2700 OWN A Fully Paid" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# look only at the two loan statuses we discussed in Chapter TODO\n", "loans_paid = loans[loans.loan_status.isin(['Fully Paid', 'Charged Off'])]\n", "# For this example, we want some paid and some charged off loans\n", "# we group them to make sure we get some of both. We then take the first 100 and remove the grouping index\n", "some_loans = loans_paid.groupby('loan_status').apply(lambda x: x.head(10)).reset_index(drop=True)\n", "# we only consider three relatively well-behaved features and the loan status\n", "small_data = some_loans[['loan_amnt', 'home_ownership', 'grade', 'loan_status']]\n", "small_data" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The loan amount is a continuous feature; it's an integer amount, but grade and home ownership are categorical. The loan_status is our classification target (which means it's also a discrete variable, but we don't consider it a feature and won't process it as such).\n", "Scikit-learn requires you to explicitly handle categorical features in most cases, which is unlike some other libraries and frameworks. However, that gives you more control over the processing, and a better idea of what happens to your data.\n", "\n", "The encoding that is most appropriate depends on the model you're using, but there are some general encoding schemes that are frequently used." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Ordinal encoding\n", "One of the simplest ways to encode categorical data is to assign an integer to each category. You could do this with the ``OrdinalEncoder`` in scikit-learn, or with pandas by using pandas categorical data:" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0 MORTGAGE\n", "1 MORTGAGE\n", "2 MORTGAGE\n", "3 RENT\n", "4 MORTGAGE\n", "5 MORTGAGE\n", "6 RENT\n", "7 OWN\n", "8 RENT\n", "9 MORTGAGE\n", "10 MORTGAGE\n", "11 MORTGAGE\n", "12 MORTGAGE\n", "13 RENT\n", "14 MORTGAGE\n", "15 RENT\n", "16 RENT\n", "17 RENT\n", "18 MORTGAGE\n", "19 OWN\n", "Name: home_ownership, dtype: category\n", "Categories (3, object): [MORTGAGE, OWN, RENT]" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# extract a column and convert it to categorical data (it was represented as strings before)\n", "home_ownership_cat = small_data.home_ownership.astype('category')\n", "home_ownership_cat" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0 0\n", "1 0\n", "2 0\n", "3 2\n", "4 0\n", "5 0\n", "6 2\n", "7 1\n", "8 2\n", "9 0\n", "10 0\n", "11 0\n", "12 0\n", "13 2\n", "14 0\n", "15 2\n", "16 2\n", "17 2\n", "18 0\n", "19 1\n", "dtype: int8" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# get integer codes from the categorical data\n", "# all categorical operations are accessible through the cat attribute:\n", "home_ownership_cat.cat.codes" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We could create a new dataframe using these integer codes, which now could be interpreted by a machine learning model. However, this is often problematic as this imposes an order and a distance between the different categories, that might not accurately reflect the semantics of the data. Both pandas and scikit-learn by default use the lexical ordering of categories, so MORTGAGE corresponds to 0, OWN to 1 and RENT to 2. This order makes little sense. We could specify our own ordering, say RENT, MORTGAGE, OWN (describing degrees of ownership) but this is also not entirely satisfactory: if we encode it using integers, we postulate that the difference between RENT and MORTGAGE is the same as the difference between MORTGAGE and OWN, and the difference between RENT and OWN is twice the distance between MORTGAGE and OWN. Making these assumption seems somewhat questionable, and in many cases, even ordering the categories might be hard - imagine working on a dataset of cars that includes the color, imposing any ordering there seems very arbitray." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "For the grades, using integer encodings might be reasonable, as there is a clear ordering and distance. Whether this is appropriate might depend on the model you are using. If there is very few categories, such as here, it's probably a safer bet to forego the ordinal encoding and use a different scheme instead." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## One-Hot Encoding\n", "\n", "The most commonly used encoding scheme for categorical variables by far is the so-called one-hot encoding or dummy encoding. The idea behind one-hot encoding is to add a new column for each value of a categorical variable, and set the column to 1 for the category that applies to the row, and 0 for all the other categories. An easy way to compute this encoding is the ``get_dummies`` function in pandas:" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
grade
0A
1C
2A
3E
4C
\n", "
" ], "text/plain": [ " grade\n", "0 A\n", "1 C\n", "2 A\n", "3 E\n", "4 C" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "small_data[['grade']].head()" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
grade_Agrade_Bgrade_Cgrade_Dgrade_E
010000
100100
210000
300001
400100
\n", "
" ], "text/plain": [ " grade_A grade_B grade_C grade_D grade_E\n", "0 1 0 0 0 0\n", "1 0 0 1 0 0\n", "2 1 0 0 0 0\n", "3 0 0 0 0 1\n", "4 0 0 1 0 0" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "pd.get_dummies(small_data[['grade']]).head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "As you can see, ``get_dummies`` replaced the one column ``grade`` by five columns, one for each possible value. The original value for the grade of first row was ``A`` so the new column ``grade_A`` has a 1, while all the other columns have a 0.\n", "We can also call ``get_dummies`` on the whole dataframe. In this case, it will apply dummy encoding to all the columns that have either categorical data or objects (including strings):" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
loan_amnthome_ownership_MORTGAGEhome_ownership_OWNhome_ownership_RENTgrade_Agrade_Bgrade_Cgrade_Dgrade_Eloan_status_Charged Offloan_status_Fully Paid
080001001000010
160001000010010
2100001001000010
3100000010000110
4350001000010010
\n", "
" ], "text/plain": [ " loan_amnt home_ownership_MORTGAGE home_ownership_OWN \\\n", "0 8000 1 0 \n", "1 6000 1 0 \n", "2 10000 1 0 \n", "3 10000 0 0 \n", "4 35000 1 0 \n", "\n", " home_ownership_RENT grade_A grade_B grade_C grade_D grade_E \\\n", "0 0 1 0 0 0 0 \n", "1 0 0 0 1 0 0 \n", "2 0 1 0 0 0 0 \n", "3 1 0 0 0 0 1 \n", "4 0 0 0 1 0 0 \n", "\n", " loan_status_Charged Off loan_status_Fully Paid \n", "0 1 0 \n", "1 1 0 \n", "2 1 0 \n", "3 1 0 \n", "4 1 0 " ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "pd.get_dummies(small_data).head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "As you can see, ``loan_amnt`` wasn't changed, while the dummy encoding was applied to all the other columns, including the target ``loan_status``. As we don't want to encode this column, we can explicitly provide the columns that we want to encode:" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
loan_amntloan_statushome_ownership_MORTGAGEhome_ownership_OWNhome_ownership_RENTgrade_Agrade_Bgrade_Cgrade_Dgrade_E
08000Charged Off10010000
16000Charged Off10000100
210000Charged Off10010000
310000Charged Off00100001
435000Charged Off10000100
\n", "
" ], "text/plain": [ " loan_amnt loan_status home_ownership_MORTGAGE home_ownership_OWN \\\n", "0 8000 Charged Off 1 0 \n", "1 6000 Charged Off 1 0 \n", "2 10000 Charged Off 1 0 \n", "3 10000 Charged Off 0 0 \n", "4 35000 Charged Off 1 0 \n", "\n", " home_ownership_RENT grade_A grade_B grade_C grade_D grade_E \n", "0 0 1 0 0 0 0 \n", "1 0 0 0 1 0 0 \n", "2 0 1 0 0 0 0 \n", "3 1 0 0 0 0 1 \n", "4 0 0 0 1 0 0 " ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "pd.get_dummies(small_data, columns=['home_ownership', 'grade']).head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "```{note}\n", "If a categorical variable was already represented as an integer you can force pandas to apply dummy encoding by passing the column name to the ``columns`` parameter of ``pd.get_dummies``.\n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Aligning dataframes with pandas\n", "A common problem in using ``get_dummies`` is that if you have multiple datasets or files, and you call ``get_dummies`` on each of them, you might get inconsistent encodings.\n", "Let's split our toy data into training and test set and apply ``get_dummies`` on them separately:" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
loan_amntloan_statushome_ownership_MORTGAGEhome_ownership_RENTgrade_Agrade_Bgrade_Cgrade_Dgrade_E
172500Fully Paid0100100
184000Fully Paid1000010
1140000Fully Paid1000100
635000Charged Off0100100
148425Fully Paid1000001
16000Charged Off1000100
210000Charged Off1010000
1220000Fully Paid1010000
310000Charged Off0100001
816000Charged Off0101000
08000Charged Off1010000
166600Fully Paid0101000
435000Charged Off1000100
1520000Fully Paid0100010
925000Charged Off1010000
\n", "
" ], "text/plain": [ " loan_amnt loan_status home_ownership_MORTGAGE home_ownership_RENT \\\n", "17 2500 Fully Paid 0 1 \n", "18 4000 Fully Paid 1 0 \n", "11 40000 Fully Paid 1 0 \n", "6 35000 Charged Off 0 1 \n", "14 8425 Fully Paid 1 0 \n", "1 6000 Charged Off 1 0 \n", "2 10000 Charged Off 1 0 \n", "12 20000 Fully Paid 1 0 \n", "3 10000 Charged Off 0 1 \n", "8 16000 Charged Off 0 1 \n", "0 8000 Charged Off 1 0 \n", "16 6600 Fully Paid 0 1 \n", "4 35000 Charged Off 1 0 \n", "15 20000 Fully Paid 0 1 \n", "9 25000 Charged Off 1 0 \n", "\n", " grade_A grade_B grade_C grade_D grade_E \n", "17 0 0 1 0 0 \n", "18 0 0 0 1 0 \n", "11 0 0 1 0 0 \n", "6 0 0 1 0 0 \n", "14 0 0 0 0 1 \n", "1 0 0 1 0 0 \n", "2 1 0 0 0 0 \n", "12 1 0 0 0 0 \n", "3 0 0 0 0 1 \n", "8 0 1 0 0 0 \n", "0 1 0 0 0 0 \n", "16 0 1 0 0 0 \n", "4 0 0 1 0 0 \n", "15 0 0 0 1 0 \n", "9 1 0 0 0 0 " ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from sklearn.model_selection import train_test_split\n", "small_train, small_test = train_test_split(small_data, random_state=21)\n", "# To avoid setting with copy warnings we copy the data after splitting\n", "# this is probably not necessary\n", "small_train = small_train.copy()\n", "small_test = small_test.copy()\n", "pd.get_dummies(small_train, columns=['home_ownership', 'grade'])" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
loan_amntloan_statushome_ownership_MORTGAGEhome_ownership_OWNhome_ownership_RENTgrade_Agrade_Bgrade_Cgrade_D
715000Charged Off0100100
1030000Fully Paid1000001
192700Fully Paid0101000
134500Fully Paid0010100
54800Charged Off1000010
\n", "
" ], "text/plain": [ " loan_amnt loan_status home_ownership_MORTGAGE home_ownership_OWN \\\n", "7 15000 Charged Off 0 1 \n", "10 30000 Fully Paid 1 0 \n", "19 2700 Fully Paid 0 1 \n", "13 4500 Fully Paid 0 0 \n", "5 4800 Charged Off 1 0 \n", "\n", " home_ownership_RENT grade_A grade_B grade_C grade_D \n", "7 0 0 1 0 0 \n", "10 0 0 0 0 1 \n", "19 0 1 0 0 0 \n", "13 1 0 1 0 0 \n", "5 0 0 0 1 0 " ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "source": [ "pd.get_dummies(small_test, columns=['home_ownership', 'grade'])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Both of the dataframes have seven columns, but the meaning of the columns is quite different. The training set doesn't have a column for ``home_ownership=OWN`` while the test set doesn't have a column for ``grade=E``.\n", "However, remember that scikit-learn doesn't know about column names in dataframes, so if you'd passed this data directly into scikit-learn, you would get meaningless predictions without knowing it! TODO callout?\n", "\n", "There's several ways to avoid this; the easiest is to use ``get_dummies`` before splitting the data, that might not be possible if new data arrives and you want to apply an existing model.\n", "Another way is to encode all categorical variables using the pandas categorical type, and specifying all the known categories:" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [], "source": [ "ownership_cats = ['MORTGAGE', 'OWN', 'RENT']\n", "grade_cats = ['A', 'B', 'C', 'D', 'E']\n", "\n", "small_test_explicit_cats = small_test.copy()\n", "small_test_explicit_cats['home_ownership'] = pd.Categorical(small_test_explicit_cats['home_ownership'], categories=ownership_cats)\n", "small_test_explicit_cats['grade'] = pd.Categorical(small_test_explicit_cats['grade'], categories=grade_cats)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now the columns are aware of all possible values, and all of them will receive a column, whether the value is present or not (not the all-zero column `grade_E`):" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
loan_amntloan_statushome_ownership_MORTGAGEhome_ownership_OWNhome_ownership_RENTgrade_Agrade_Bgrade_Cgrade_Dgrade_E
715000Charged Off01001000
1030000Fully Paid10000010
192700Fully Paid01010000
134500Fully Paid00101000
54800Charged Off10000100
\n", "
" ], "text/plain": [ " loan_amnt loan_status home_ownership_MORTGAGE home_ownership_OWN \\\n", "7 15000 Charged Off 0 1 \n", "10 30000 Fully Paid 1 0 \n", "19 2700 Fully Paid 0 1 \n", "13 4500 Fully Paid 0 0 \n", "5 4800 Charged Off 1 0 \n", "\n", " home_ownership_RENT grade_A grade_B grade_C grade_D grade_E \n", "7 0 0 1 0 0 0 \n", "10 0 0 0 0 1 0 \n", "19 0 1 0 0 0 0 \n", "13 1 0 1 0 0 0 \n", "5 0 0 0 1 0 0 " ] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ "pd.get_dummies(small_test_explicit_cats, columns=['home_ownership', 'grade'])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This method is very explicit and safe, but can be a bit cumbersome if there's many categorical columns, or not all of the categories are known beforehand.\n", "A somewhat simpler and less explicit method is using ``pd.align`` after calling ``get_dummies``:" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
loan_amntloan_statushome_ownership_MORTGAGEhome_ownership_RENTgrade_Agrade_Bgrade_Cgrade_Dgrade_E
715000Charged Off0001000
1030000Fully Paid1000010
192700Fully Paid0010000
134500Fully Paid0101000
54800Charged Off1000100
\n", "
" ], "text/plain": [ " loan_amnt loan_status home_ownership_MORTGAGE home_ownership_RENT \\\n", "7 15000 Charged Off 0 0 \n", "10 30000 Fully Paid 1 0 \n", "19 2700 Fully Paid 0 0 \n", "13 4500 Fully Paid 0 1 \n", "5 4800 Charged Off 1 0 \n", "\n", " grade_A grade_B grade_C grade_D grade_E \n", "7 0 1 0 0 0 \n", "10 0 0 0 1 0 \n", "19 1 0 0 0 0 \n", "13 0 1 0 0 0 \n", "5 0 0 1 0 0 " ] }, "execution_count": 16, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# compute dummy encoding (the columns will not match afterwards)\n", "small_train_dummies = pd.get_dummies(small_train, columns=['home_ownership', 'grade'])\n", "small_test_dummies = pd.get_dummies(small_test, columns=['home_ownership', 'grade'])\n", "\n", "# align dataframes\n", "# join='right' aligns test (left=self) to train (right=other) keeping only the columns in train\n", "# axis=1 means we align only the columns, and don't try joining the row indices\n", "# align returns two aligned frames; because we did a right join, train is unchanged\n", "# so we can discard the second return value (and assign it to _)\n", "# fill value specifies what to put into previously non-existing columns\n", "test_aligned, _ = small_test_dummies.align(small_train_dummies, join='right', axis=1, fill_value=0)\n", "test_aligned" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The result is an aligned dataframe ``test_aligned`` that has the same columns as ``small_train_dummies``, the dataframe it was aligned with.\n", "This ensures that the shapes are compatible between training and test set and we will get meaningful results from scikit-learn.\n", "Note that 'OWN' column that was previously present in the test set was dropped as it is not present in the training set.\n", "We could also perform an inner join when aligning the dataframes (which is the default in ``pd.align`` if you don't specify ``join``), so that the aligned dataframes contain all the columns present in either of the dataframes:\n" ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
grade_Agrade_Bgrade_Cgrade_Dgrade_Ehome_ownership_MORTGAGEhome_ownership_OWNhome_ownership_RENTloan_amntloan_status
70100001015000Charged Off
100001010030000Fully Paid
19100000102700Fully Paid
13010000014500Fully Paid
5001001004800Charged Off
\n", "
" ], "text/plain": [ " grade_A grade_B grade_C grade_D grade_E home_ownership_MORTGAGE \\\n", "7 0 1 0 0 0 0 \n", "10 0 0 0 1 0 1 \n", "19 1 0 0 0 0 0 \n", "13 0 1 0 0 0 0 \n", "5 0 0 1 0 0 1 \n", "\n", " home_ownership_OWN home_ownership_RENT loan_amnt loan_status \n", "7 1 0 15000 Charged Off \n", "10 0 0 30000 Fully Paid \n", "19 1 0 2700 Fully Paid \n", "13 0 1 4500 Fully Paid \n", "5 0 0 4800 Charged Off " ] }, "execution_count": 17, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# if we don't specify join, an inner join is performed and we retain all the columns\n", "# in this case we also want to store the new aligned training set\n", "test_aligned, train_aligned = small_test_dummies.align(small_train_dummies, axis=1, fill_value=0)\n", "test_aligned" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "However, that's not very useful if you already created a model using the original training dataset." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## One-hot with sklearn\n", "Another option (and the one we will use for most of the book) is to do our encoding with scikit-learn. Because scikit-learn has a concept of training and test set, the issue of aligning the data is handled automatically.\n", "The dummy or one-hot encoding in scikit-learn is implemented in the ``OneHotEncoder``, which is a transformer, just like other preprocessing methods." ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [], "source": [ "from sklearn.preprocessing import OneHotEncoder\n", "# By default, OneHotEncoder errors if it sees unknown categories in the test data\n", "# we will overwrite this behavior by specifying handle_unknown='ignore'\n", "# also OneHotEncoder by default outputs scipy sparse matrices, which are more efficient but cumbersome\n", "# we disable that with sparse=False\n", "ohe = OneHotEncoder(handle_unknown='ignore', sparse=False)\n", "# fit on the training set; stores the categories for each column\n", "ohe.fit(small_train)\n", "# apply one-hot encoding to both training and test set\n", "X_train_ohe = ohe.transform(small_train)\n", "X_test_ohe = ohe.transform(small_test)" ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 0., 0.,\n", " 1., 0., 0., 0., 1.],\n", " [0., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 0., 0., 0.,\n", " 0., 1., 0., 0., 1.],\n", " [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 0.,\n", " 1., 0., 0., 0., 1.],\n", " [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 0., 0., 1., 0., 0.,\n", " 1., 0., 0., 1., 0.],\n", " [0., 0., 0., 0., 0., 1., 0., 0., 0., 0., 0., 0., 1., 0., 0., 0.,\n", " 0., 0., 1., 0., 1.],\n", " [0., 0., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 0., 0., 0.,\n", " 1., 0., 0., 1., 0.],\n", " [0., 0., 0., 0., 0., 0., 1., 0., 0., 0., 0., 0., 1., 0., 1., 0.,\n", " 0., 0., 0., 1., 0.],\n", " [0., 0., 0., 0., 0., 0., 0., 0., 1., 0., 0., 0., 1., 0., 1., 0.,\n", " 0., 0., 0., 0., 1.],\n", " [0., 0., 0., 0., 0., 0., 1., 0., 0., 0., 0., 0., 0., 1., 0., 0.,\n", " 0., 0., 1., 1., 0.],\n", " [0., 0., 0., 0., 0., 0., 0., 1., 0., 0., 0., 0., 0., 1., 0., 1.,\n", " 0., 0., 0., 1., 0.],\n", " [0., 0., 0., 0., 1., 0., 0., 0., 0., 0., 0., 0., 1., 0., 1., 0.,\n", " 0., 0., 0., 1., 0.],\n", " [0., 0., 0., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 0., 1.,\n", " 0., 0., 0., 0., 1.],\n", " [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 0., 1., 0., 0., 0.,\n", " 1., 0., 0., 1., 0.],\n", " [0., 0., 0., 0., 0., 0., 0., 0., 1., 0., 0., 0., 0., 1., 0., 0.,\n", " 0., 1., 0., 0., 1.],\n", " [0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 0., 0., 1., 0., 1., 0.,\n", " 0., 0., 0., 1., 0.]])" ] }, "execution_count": 19, "metadata": {}, "output_type": "execute_result" } ], "source": [ "X_train_ohe" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "```{margin} Feature names in scikit-learn\n", "While scikit-learn doesn't output pandas dataframes (for now) we can get feature names for the output of some transformers using the ``get_feature_names`` method.\n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "There are two things you might notice in this output: as with all scikit-learn transformations, the output is a numpy array without column names, which can be a bit inconvenient.\n", "Secondly, all the columns have been one-hot-encoded, the loan amount is not present any more.\n", "We can actually get the feature names by using the ``get_feature_names`` method of the ``OneHotEncoder``:" ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array(['x0_2500', 'x0_4000', 'x0_6000', 'x0_6600', 'x0_8000', 'x0_8425',\n", " 'x0_10000', 'x0_16000', 'x0_20000', 'x0_25000', 'x0_35000',\n", " 'x0_40000', 'x1_MORTGAGE', 'x1_RENT', 'x2_A', 'x2_B', 'x2_C',\n", " 'x2_D', 'x2_E', 'x3_Charged Off', 'x3_Fully Paid'], dtype=object)" ] }, "execution_count": 20, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ohe.get_feature_names()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "By default, the input columns in scikit-learn are named ``x0``, ``x1`` and so on, so what this tells us is that for the first feature, loan amount, several new columns were added to one-hot-encode the observed integer values, which was not what we had in mind. We can get more informative feature names by passing the original dataframe column names to ``get_feature_names``:" ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array(['loan_amnt_2500', 'loan_amnt_4000', 'loan_amnt_6000',\n", " 'loan_amnt_6600', 'loan_amnt_8000', 'loan_amnt_8425',\n", " 'loan_amnt_10000', 'loan_amnt_16000', 'loan_amnt_20000',\n", " 'loan_amnt_25000', 'loan_amnt_35000', 'loan_amnt_40000',\n", " 'home_ownership_MORTGAGE', 'home_ownership_RENT', 'grade_A',\n", " 'grade_B', 'grade_C', 'grade_D', 'grade_E',\n", " 'loan_status_Charged Off', 'loan_status_Fully Paid'], dtype=object)" ] }, "execution_count": 21, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ohe.get_feature_names(small_train.columns)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "OneHotEncoder, like all other estimators in scikit-learn, always works on all input columns. So we need to pass it only the column that we want to transform.\n", "So one possible way to do this would be to slice of the categorical columns, transform them with ``OneHotEncoder``, like so:" ] }, { "cell_type": "code", "execution_count": 22, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Index(['loan_amnt', 'home_ownership', 'grade', 'loan_status'], dtype='object')" ] }, "execution_count": 22, "metadata": {}, "output_type": "execute_result" } ], "source": [ "small_train.columns" ] }, { "cell_type": "code", "execution_count": 23, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[0., 1., 0., 0., 1., 0., 0.],\n", " [1., 0., 0., 0., 0., 1., 0.],\n", " [1., 0., 0., 0., 1., 0., 0.],\n", " [0., 1., 0., 0., 1., 0., 0.],\n", " [1., 0., 0., 0., 0., 0., 1.],\n", " [1., 0., 0., 0., 1., 0., 0.],\n", " [1., 0., 1., 0., 0., 0., 0.],\n", " [1., 0., 1., 0., 0., 0., 0.],\n", " [0., 1., 0., 0., 0., 0., 1.],\n", " [0., 1., 0., 1., 0., 0., 0.],\n", " [1., 0., 1., 0., 0., 0., 0.],\n", " [0., 1., 0., 1., 0., 0., 0.],\n", " [1., 0., 0., 0., 1., 0., 0.],\n", " [0., 1., 0., 0., 0., 1., 0.],\n", " [1., 0., 1., 0., 0., 0., 0.]])" ] }, "execution_count": 23, "metadata": {}, "output_type": "execute_result" } ], "source": [ "train_cat = small_train[['home_ownership', 'grade']]\n", "ohe = OneHotEncoder(handle_unknown='ignore', sparse=False)\n", "train_cat_ohe = ohe.fit_transform(train_cat)\n", "train_cat_ohe" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can now make this one-hot-encoded array back into a dataframe by using ``get_feature_names``:" ] }, { "cell_type": "code", "execution_count": 24, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
home_ownership_MORTGAGEhome_ownership_RENTgrade_Agrade_Bgrade_Cgrade_Dgrade_E
170.01.00.00.01.00.00.0
181.00.00.00.00.01.00.0
111.00.00.00.01.00.00.0
60.01.00.00.01.00.00.0
141.00.00.00.00.00.01.0
\n", "
" ], "text/plain": [ " home_ownership_MORTGAGE home_ownership_RENT grade_A grade_B grade_C \\\n", "17 0.0 1.0 0.0 0.0 1.0 \n", "18 1.0 0.0 0.0 0.0 0.0 \n", "11 1.0 0.0 0.0 0.0 1.0 \n", "6 0.0 1.0 0.0 0.0 1.0 \n", "14 1.0 0.0 0.0 0.0 0.0 \n", "\n", " grade_D grade_E \n", "17 0.0 0.0 \n", "18 1.0 0.0 \n", "11 0.0 0.0 \n", "6 0.0 0.0 \n", "14 0.0 1.0 " ] }, "execution_count": 24, "metadata": {}, "output_type": "execute_result" } ], "source": [ "train_df_ohe = pd.DataFrame(train_cat_ohe,\n", " # create column names\n", " columns=ohe.get_feature_names(train_cat.columns),\n", " # keep the old index.\n", " # this is optional but will make joining with the other features easier\n", " index=train_cat.index)\n", "train_df_ohe.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Then we can concatenate it again with the remaining 'amount' feature:" ] }, { "cell_type": "code", "execution_count": 25, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
home_ownership_MORTGAGEhome_ownership_RENTgrade_Agrade_Bgrade_Cgrade_Dgrade_Eloan_amnt
170.01.00.00.01.00.00.02500
181.00.00.00.00.01.00.04000
111.00.00.00.01.00.00.040000
60.01.00.00.01.00.00.035000
141.00.00.00.00.00.01.08425
\n", "
" ], "text/plain": [ " home_ownership_MORTGAGE home_ownership_RENT grade_A grade_B grade_C \\\n", "17 0.0 1.0 0.0 0.0 1.0 \n", "18 1.0 0.0 0.0 0.0 0.0 \n", "11 1.0 0.0 0.0 0.0 1.0 \n", "6 0.0 1.0 0.0 0.0 1.0 \n", "14 1.0 0.0 0.0 0.0 0.0 \n", "\n", " grade_D grade_E loan_amnt \n", "17 0.0 0.0 2500 \n", "18 1.0 0.0 4000 \n", "11 0.0 0.0 40000 \n", "6 0.0 0.0 35000 \n", "14 0.0 1.0 8425 " ] }, "execution_count": 25, "metadata": {}, "output_type": "execute_result" } ], "source": [ "train_df_all = pd.concat([train_df_ohe, small_train[['loan_amnt']]], axis=1)\n", "train_df_all.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "While this is the result we wanted, this was pretty complicated compared to using ``pd.get_dummies``.\n", "Luckily, scikit-learn has another tool that will make this much easier, the ``ColumnTransformer``." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Selecting Columns in sklearn\n", "\n", "The ``ColumnTransformer`` is another meta-estimator, similar to the ``Pipeline``, for combining multiple transformations.\n", "In particular, the ``ColumnTransformer`` allows you to apply different transformations to different subsets of the columns.\n", "It's the only part of scikit-learn that explicitly uses pandas column names, and is made specificly to ease the use of pandas dataframes\n", "as input to scikit-learn models.\n", "\n", "In contrast to ``Pipeline``, which basically applies several transformations in sequence, the ``ColumnTransformer`` applies several transformations in parallel,\n", "each on a subset of columns, and then concatenates the results, similar to what we did manually above. This is illustrate in Figure TODO.\n", "\n", "TODO new image\n", "\n", "![:scale 100%](images/column_transformer_schematic.png)\n", "\n", "As with the pipeline, there is a ``make_column_transformer`` helper function to easily create a new column transformer.\n", "The function takes tuples, where each tuple consists of a transformation and the columns the transformation should be applied to.\n", "So to apply the ``OneHotEncoder`` to only the ``home_ownership`` and ``grade`` columns, we could do:" ] }, { "cell_type": "code", "execution_count": 26, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "ColumnTransformer(transformers=[('onehotencoder',\n", " OneHotEncoder(handle_unknown='ignore',\n", " sparse=False),\n", " ['home_ownership', 'grade'])])" ] }, "execution_count": 26, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from sklearn.compose import make_column_transformer\n", "# a single transformation, OneHotEncoder, applied to two columns\n", "ct = make_column_transformer((OneHotEncoder(handle_unknown='ignore', sparse=False), ['home_ownership', 'grade']))\n", "# now we can pass the full dataset, as the column transformer will do the subsetting for us:\n", "ct.fit(small_train)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The ``make_column_transformer`` has created ``ColumnTransformer`` object for us with a single transformer. It has also automatically generated a name for the transformer using the lower cased class name, `'onehotencoder'`.\n", "We can now use the ``ColumnTransformer`` to transform our dataset:" ] }, { "cell_type": "code", "execution_count": 27, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[0., 1., 0., 0., 1., 0., 0.],\n", " [1., 0., 0., 0., 0., 1., 0.],\n", " [1., 0., 0., 0., 1., 0., 0.],\n", " [0., 1., 0., 0., 1., 0., 0.],\n", " [1., 0., 0., 0., 0., 0., 1.],\n", " [1., 0., 0., 0., 1., 0., 0.],\n", " [1., 0., 1., 0., 0., 0., 0.],\n", " [1., 0., 1., 0., 0., 0., 0.],\n", " [0., 1., 0., 0., 0., 0., 1.],\n", " [0., 1., 0., 1., 0., 0., 0.],\n", " [1., 0., 1., 0., 0., 0., 0.],\n", " [0., 1., 0., 1., 0., 0., 0.],\n", " [1., 0., 0., 0., 1., 0., 0.],\n", " [0., 1., 0., 0., 0., 1., 0.],\n", " [1., 0., 1., 0., 0., 0., 0.]])" ] }, "execution_count": 27, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ct.transform(small_train)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Again, we can get the column names for the output using ``get_feature_names``:\n", "TODO this doesn't take the columns?!" ] }, { "cell_type": "code", "execution_count": 28, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['onehotencoder__x0_MORTGAGE',\n", " 'onehotencoder__x0_RENT',\n", " 'onehotencoder__x1_A',\n", " 'onehotencoder__x1_B',\n", " 'onehotencoder__x1_C',\n", " 'onehotencoder__x1_D',\n", " 'onehotencoder__x1_E']" ] }, "execution_count": 28, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ct.get_feature_names()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "As you can see, by default, the ``ColumnTransformer`` only keeps the columns that we mentioned.\n", "If we want to keep the remaining columns, we can specify ``remainder='passthrough':" ] }, { "cell_type": "code", "execution_count": 29, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[0.0, 1.0, 0.0, 0.0, 1.0, 0.0, 0.0, 2500, 'Fully Paid'],\n", " [1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 4000, 'Fully Paid'],\n", " [1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 40000, 'Fully Paid'],\n", " [0.0, 1.0, 0.0, 0.0, 1.0, 0.0, 0.0, 35000, 'Charged Off'],\n", " [1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 8425, 'Fully Paid'],\n", " [1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 6000, 'Charged Off'],\n", " [1.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 10000, 'Charged Off'],\n", " [1.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 20000, 'Fully Paid'],\n", " [0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 10000, 'Charged Off'],\n", " [0.0, 1.0, 0.0, 1.0, 0.0, 0.0, 0.0, 16000, 'Charged Off'],\n", " [1.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 8000, 'Charged Off'],\n", " [0.0, 1.0, 0.0, 1.0, 0.0, 0.0, 0.0, 6600, 'Fully Paid'],\n", " [1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 35000, 'Charged Off'],\n", " [0.0, 1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 20000, 'Fully Paid'],\n", " [1.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 25000, 'Charged Off']],\n", " dtype=object)" ] }, "execution_count": 29, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ct = make_column_transformer((OneHotEncoder(handle_unknown='ignore', sparse=False),\n", " ['home_ownership', 'grade']),\n", " remainder='passthrough'\n", " )\n", "ct.fit_transform(small_train)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Typically we would not want the target in our data within scikit-learn, though, so we might want do drop it before, or we might want to specify only to pass through the 'loan_amnt' column.\n", "We can pass through only some columns by adding a new transformation in the ColumnTransformer, though instead of passing a scikit-learn transformer, we pass the string `'passthrough'`:" ] }, { "cell_type": "code", "execution_count": 30, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[ 0., 1., 0., 0., 1., 0., 0., 2500.],\n", " [ 1., 0., 0., 0., 0., 1., 0., 4000.],\n", " [ 1., 0., 0., 0., 1., 0., 0., 40000.],\n", " [ 0., 1., 0., 0., 1., 0., 0., 35000.],\n", " [ 1., 0., 0., 0., 0., 0., 1., 8425.],\n", " [ 1., 0., 0., 0., 1., 0., 0., 6000.],\n", " [ 1., 0., 1., 0., 0., 0., 0., 10000.],\n", " [ 1., 0., 1., 0., 0., 0., 0., 20000.],\n", " [ 0., 1., 0., 0., 0., 0., 1., 10000.],\n", " [ 0., 1., 0., 1., 0., 0., 0., 16000.],\n", " [ 1., 0., 1., 0., 0., 0., 0., 8000.],\n", " [ 0., 1., 0., 1., 0., 0., 0., 6600.],\n", " [ 1., 0., 0., 0., 1., 0., 0., 35000.],\n", " [ 0., 1., 0., 0., 0., 1., 0., 20000.],\n", " [ 1., 0., 1., 0., 0., 0., 0., 25000.]])" ] }, "execution_count": 30, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ct = make_column_transformer((OneHotEncoder(handle_unknown='ignore', sparse=False),\n", " ['home_ownership', 'grade']),\n", " ('passthrough', ['loan_amnt'])\n", " )\n", "# we set numpy printoptions to not use scientific notation for nicer output\n", "# and suppress all but three decimal places\n", "import numpy as np\n", "np.set_printoptions(suppress=True, precision=3)\n", "ct.fit_transform(small_train)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If instead, we want to scale the 'loan_amnt' column, we can pass ``StandardScaler`` instead of ``'passthrough'``:\n" ] }, { "cell_type": "code", "execution_count": 31, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[ 0. , 1. , 0. , 0. , 1. , 0. , 0. , -1.173],\n", " [ 1. , 0. , 0. , 0. , 0. , 1. , 0. , -1.047],\n", " [ 1. , 0. , 0. , 0. , 1. , 0. , 0. , 1.984],\n", " [ 0. , 1. , 0. , 0. , 1. , 0. , 0. , 1.563],\n", " [ 1. , 0. , 0. , 0. , 0. , 0. , 1. , -0.674],\n", " [ 1. , 0. , 0. , 0. , 1. , 0. , 0. , -0.879],\n", " [ 1. , 0. , 1. , 0. , 0. , 0. , 0. , -0.542],\n", " [ 1. , 0. , 1. , 0. , 0. , 0. , 0. , 0.3 ],\n", " [ 0. , 1. , 0. , 0. , 0. , 0. , 1. , -0.542],\n", " [ 0. , 1. , 0. , 1. , 0. , 0. , 0. , -0.037],\n", " [ 1. , 0. , 1. , 0. , 0. , 0. , 0. , -0.71 ],\n", " [ 0. , 1. , 0. , 1. , 0. , 0. , 0. , -0.828],\n", " [ 1. , 0. , 0. , 0. , 1. , 0. , 0. , 1.563],\n", " [ 0. , 1. , 0. , 0. , 0. , 1. , 0. , 0.3 ],\n", " [ 1. , 0. , 1. , 0. , 0. , 0. , 0. , 0.721]])" ] }, "execution_count": 31, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from sklearn.preprocessing import StandardScaler\n", "ct = make_column_transformer((OneHotEncoder(handle_unknown='ignore', sparse=False),\n", " ['home_ownership', 'grade']),\n", " (StandardScaler(), ['loan_amnt'])\n", " )\n", "ct.fit_transform(small_train)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now we can finally transform our test dataset without much work:" ] }, { "cell_type": "code", "execution_count": 32, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[ 0. , 0. , 0. , 1. , 0. , 0. , 0. , -0.121],\n", " [ 1. , 0. , 0. , 0. , 0. , 1. , 0. , 1.142],\n", " [ 0. , 0. , 1. , 0. , 0. , 0. , 0. , -1.156],\n", " [ 0. , 1. , 0. , 1. , 0. , 0. , 0. , -1.005],\n", " [ 1. , 0. , 0. , 0. , 1. , 0. , 0. , -0.98 ]])" ] }, "execution_count": 32, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ct.transform(small_test)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Instead of using the ``make_column_transformer`` function, we can also directly use the ``ColumnTransformer`` class. As with the ``Pipeline``, this allows us to explicitly give names to the different transformers:" ] }, { "cell_type": "code", "execution_count": 33, "metadata": {}, "outputs": [], "source": [ "from sklearn.compose import ColumnTransformer\n", "# this is equivalent to the the column transformer created above\n", "# each transformation is a tuple (name, transformer, columns)\n", "ct = ColumnTransformer([('ohe', OneHotEncoder(handle_unknown='ignore', sparse=False),\n", " ['home_ownership', 'grade']),\n", " ('scaler', StandardScaler(), ['loan_amnt'])])" ] }, { "cell_type": "code", "execution_count": 34, "metadata": { "tags": [ "remove-input" ] }, "outputs": [ { "data": { "text/html": [ "
ColumnTransformer(transformers=[('onehotencoder',\n",
       "                                 OneHotEncoder(handle_unknown='ignore',\n",
       "                                               sparse=False),\n",
       "                                 ['home_ownership', 'grade']),\n",
       "                                ('standardscaler', StandardScaler(),\n",
       "                                 ['loan_amnt'])])
['home_ownership', 'grade']
OneHotEncoder(handle_unknown='ignore', sparse=False)
['loan_amnt']
StandardScaler()
" ], "text/plain": [ "ColumnTransformer(transformers=[('onehotencoder',\n", " OneHotEncoder(handle_unknown='ignore',\n", " sparse=False),\n", " ['home_ownership', 'grade']),\n", " ('standardscaler', StandardScaler(),\n", " ['loan_amnt'])])" ] }, "metadata": { "scrapbook": { "mime_prefix": "", "name": "ct_diagram" } }, "output_type": "display_data" }, { "data": { "text/html": [ "
ColumnTransformer(transformers=[('onehotencoder',\n",
       "                                 OneHotEncoder(handle_unknown='ignore',\n",
       "                                               sparse=False),\n",
       "                                 ['home_ownership', 'grade']),\n",
       "                                ('standardscaler', StandardScaler(),\n",
       "                                 ['loan_amnt'])])
['home_ownership', 'grade']
OneHotEncoder(handle_unknown='ignore', sparse=False)
['loan_amnt']
StandardScaler()
" ], "text/plain": [ "ColumnTransformer(transformers=[('onehotencoder',\n", " OneHotEncoder(handle_unknown='ignore',\n", " sparse=False),\n", " ['home_ownership', 'grade']),\n", " ('standardscaler', StandardScaler(),\n", " ['loan_amnt'])])" ] }, "execution_count": 34, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# hidden for gluing\n", "import sklearn\n", "sklearn.set_config(display='diagram')\n", "ct = make_column_transformer((OneHotEncoder(handle_unknown='ignore', sparse=False),\n", " ['home_ownership', 'grade']),\n", " (StandardScaler(), ['loan_amnt']))\n", "from myst_nb import glue\n", "glue('ct_diagram', ct)\n", "ct" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "````{note} Diagram representations of estimators\n", "Starting with version 0.23, scikit-learn has diagram representations for estimators when running in jupyter.\n", "These can be enabled using ``sklearn.set_config``:\n", "\n", "```python\n", "import sklearn\n", "sklearn.set_config(display='diagram')\n", "```\n", "\n", "```{glue:} ct_diagram\n", "```\n", "\n", "In particular for more complex workflows they can come in really handy. Clicking on any of the estimators in the diagram will provide you with more details.\n", "````" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Combining ColumnTransformer and Pipeline\n", "\n", "While ``ColumnTransformer`` by itself is already quite awesome, the real power comes from combining it with ``Pipeline`` to encapsulate the whole preprocessing and model training.\n", "Let's apply ``KNeighborsClassifier`` to the small subset of the lending club data we have been looking at. This is more for illustrative purposes, as we're using a tiny subset, and KNeighborsClassifier is potentially not a good model for this dataset, but we will see this combination in many of our later examples.\n", "\n", "Let's start from the beginning create a small dataset with 200 'Charged Off' and 200 'Fully Paid' loans." ] }, { "cell_type": "code", "execution_count": 35, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
loan_amnthome_ownershipgradeloan_status
08000MORTGAGEACharged Off
16000MORTGAGECCharged Off
210000MORTGAGEACharged Off
310000RENTECharged Off
435000MORTGAGECCharged Off
\n", "
" ], "text/plain": [ " loan_amnt home_ownership grade loan_status\n", "0 8000 MORTGAGE A Charged Off\n", "1 6000 MORTGAGE C Charged Off\n", "2 10000 MORTGAGE A Charged Off\n", "3 10000 RENT E Charged Off\n", "4 35000 MORTGAGE C Charged Off" ] }, "execution_count": 35, "metadata": {}, "output_type": "execute_result" } ], "source": [ "loans = pd.read_csv(\"C:/Users/t3kci/Downloads/loan.csv/loan.csv\", nrows=1000000, low_memory=False)\n", "loans_paid = loans[loans.loan_status.isin(['Fully Paid', 'Charged Off'])]\n", "some_loans = loans_paid.groupby('loan_status').apply(lambda x: x.head(200)).reset_index(drop=True)\n", "X = some_loans[['loan_amnt', 'home_ownership', 'grade', 'loan_status']]\n", "X.head()" ] }, { "cell_type": "code", "execution_count": 36, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Fully Paid 200\n", "Charged Off 200\n", "Name: loan_status, dtype: int64" ] }, "execution_count": 36, "metadata": {}, "output_type": "execute_result" } ], "source": [ "X.loan_status.value_counts()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "First, we split off the target column as usual, and split the data into training and test set:" ] }, { "cell_type": "code", "execution_count": 37, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
loan_amnthome_ownershipgrade
33212000MORTGAGEE
617125RENTE
18510500OWND
7515000OWNC
13540000OWNA
\n", "
" ], "text/plain": [ " loan_amnt home_ownership grade\n", "332 12000 MORTGAGE E\n", "61 7125 RENT E\n", "185 10500 OWN D\n", "75 15000 OWN C\n", "135 40000 OWN A" ] }, "execution_count": 37, "metadata": {}, "output_type": "execute_result" } ], "source": [ "X_train, X_test, y_train, y_test = train_test_split(X.drop(columns='loan_status'),\n", " X.loan_status, random_state=23)\n", "\n", "X_train.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Next, we assemble our ``ColumnTransformer`` as we did above, applying scaling to the ``loan_amnt`` and applying one-hot encoding to ``home_ownership`` and ``grade``:" ] }, { "cell_type": "code", "execution_count": 38, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
ColumnTransformer(transformers=[('onehotencoder',\n",
       "                                 OneHotEncoder(handle_unknown='ignore',\n",
       "                                               sparse=False),\n",
       "                                 ['home_ownership', 'grade']),\n",
       "                                ('standardscaler', StandardScaler(),\n",
       "                                 ['loan_amnt'])])
['home_ownership', 'grade']
OneHotEncoder(handle_unknown='ignore', sparse=False)
['loan_amnt']
StandardScaler()
" ], "text/plain": [ "ColumnTransformer(transformers=[('onehotencoder',\n", " OneHotEncoder(handle_unknown='ignore',\n", " sparse=False),\n", " ['home_ownership', 'grade']),\n", " ('standardscaler', StandardScaler(),\n", " ['loan_amnt'])])" ] }, "execution_count": 38, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ct = make_column_transformer((OneHotEncoder(handle_unknown='ignore', sparse=False),\n", " ['home_ownership', 'grade']),\n", " (StandardScaler(), ['loan_amnt']))\n", "ct" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now, we use this ``ColumnTransformer`` as a preprocessing step in a ``Pipeline`` with ``KNeighborsClassifier``:" ] }, { "cell_type": "code", "execution_count": 39, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
Pipeline(steps=[('columntransformer',\n",
       "                 ColumnTransformer(transformers=[('onehotencoder',\n",
       "                                                  OneHotEncoder(handle_unknown='ignore',\n",
       "                                                                sparse=False),\n",
       "                                                  ['home_ownership', 'grade']),\n",
       "                                                 ('standardscaler',\n",
       "                                                  StandardScaler(),\n",
       "                                                  ['loan_amnt'])])),\n",
       "                ('kneighborsclassifier', KNeighborsClassifier(n_neighbors=1))])
ColumnTransformer(transformers=[('onehotencoder',\n",
       "                                 OneHotEncoder(handle_unknown='ignore',\n",
       "                                               sparse=False),\n",
       "                                 ['home_ownership', 'grade']),\n",
       "                                ('standardscaler', StandardScaler(),\n",
       "                                 ['loan_amnt'])])
['home_ownership', 'grade']
OneHotEncoder(handle_unknown='ignore', sparse=False)
['loan_amnt']
StandardScaler()
KNeighborsClassifier(n_neighbors=1)
" ], "text/plain": [ "Pipeline(steps=[('columntransformer',\n", " ColumnTransformer(transformers=[('onehotencoder',\n", " OneHotEncoder(handle_unknown='ignore',\n", " sparse=False),\n", " ['home_ownership', 'grade']),\n", " ('standardscaler',\n", " StandardScaler(),\n", " ['loan_amnt'])])),\n", " ('kneighborsclassifier', KNeighborsClassifier(n_neighbors=1))])" ] }, "execution_count": 39, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from sklearn.pipeline import make_pipeline\n", "from sklearn.neighbors import KNeighborsClassifier\n", "\n", "pipe = make_pipeline(ct, KNeighborsClassifier(n_neighbors=1))\n", "pipe" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now, everything we need to do is embedded in the ``pipe`` object, and we can just run ``fit`` on the training set and ``score`` on the test set:" ] }, { "cell_type": "code", "execution_count": 40, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0.59" ] }, "execution_count": 40, "metadata": {}, "output_type": "execute_result" } ], "source": [ "pipe.fit(X_train, y_train)\n", "pipe.score(X_test, y_test)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The result here is not too overwhelming, but a bit better than chance (which would be 50%, given that we constructed a balanced dataset).\n", "However, the overall code for building and evaluating the model was quite minimal, and leaves little opportunity for casual mistakes." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Selecting columns by type\n", "If there are many columns, such as in the full lending club data, it can sometimes be a challenge to determine the types and correct preprocessing for all of them.\n", "Usually, it pays of to understand each individual column, and invest some time understanding the meaning of all the values, plotting them, and so on.\n", "For example, for integer values it might not be obvious whether they should be one-hot-encoded or not. The annual salary amount in the lending club data is continuous, but the easiest way to actually be certain about that is to know what the data represents. We know salary, or really any amount of money, will be continuous. Luckily in this dataset all the categorical variable are represented as strings, and so it's relatively easy to understand what they mean.\n", "But without additional information, if a column contains small integers, these could either correspond to a continuous quantity, or be an encoding for categories.\n", "\n", "This somewhat tricky case aside, for some quick plotting or prototyping, it might sometimes be enough to look at just the dtype of the columns.\n", "Let's look again at the dtypes of the full lending club data:" ] }, { "cell_type": "code", "execution_count": 41, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "float64 70\n", "int64 39\n", "object 36\n", "dtype: int64" ] }, "execution_count": 41, "metadata": {}, "output_type": "execute_result" } ], "source": [ "loans.dtypes.value_counts()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We could select only column of a certain type by using boolean masks:" ] }, { "cell_type": "code", "execution_count": 42, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
termgradesub_gradeemp_titleemp_lengthhome_ownershipverification_statusissue_dloan_statuspymnt_plan...hardship_statushardship_start_datehardship_end_datepayment_plan_start_datehardship_loan_statusdisbursement_methoddebt_settlement_flagdebt_settlement_flag_datesettlement_statussettlement_date
036 monthsCC1Chef10+ yearsRENTNot VerifiedDec-2018Currentn...NaNNaNNaNNaNNaNCashNNaNNaNNaN
160 monthsDD2Postmaster10+ yearsMORTGAGESource VerifiedDec-2018Currentn...NaNNaNNaNNaNNaNCashNNaNNaNNaN
236 monthsDD1Administrative6 yearsMORTGAGESource VerifiedDec-2018Currentn...NaNNaNNaNNaNNaNCashNNaNNaNNaN
336 monthsDD2IT Supervisor10+ yearsMORTGAGESource VerifiedDec-2018Currentn...NaNNaNNaNNaNNaNCashNNaNNaNNaN
460 monthsCC4Mechanic10+ yearsMORTGAGENot VerifiedDec-2018Currentn...NaNNaNNaNNaNNaNCashNNaNNaNNaN
\n", "

5 rows × 36 columns

\n", "
" ], "text/plain": [ " term grade sub_grade emp_title emp_length home_ownership \\\n", "0 36 months C C1 Chef 10+ years RENT \n", "1 60 months D D2 Postmaster 10+ years MORTGAGE \n", "2 36 months D D1 Administrative 6 years MORTGAGE \n", "3 36 months D D2 IT Supervisor 10+ years MORTGAGE \n", "4 60 months C C4 Mechanic 10+ years MORTGAGE \n", "\n", " verification_status issue_d loan_status pymnt_plan ... hardship_status \\\n", "0 Not Verified Dec-2018 Current n ... NaN \n", "1 Source Verified Dec-2018 Current n ... NaN \n", "2 Source Verified Dec-2018 Current n ... NaN \n", "3 Source Verified Dec-2018 Current n ... NaN \n", "4 Not Verified Dec-2018 Current n ... NaN \n", "\n", " hardship_start_date hardship_end_date payment_plan_start_date \\\n", "0 NaN NaN NaN \n", "1 NaN NaN NaN \n", "2 NaN NaN NaN \n", "3 NaN NaN NaN \n", "4 NaN NaN NaN \n", "\n", " hardship_loan_status disbursement_method debt_settlement_flag \\\n", "0 NaN Cash N \n", "1 NaN Cash N \n", "2 NaN Cash N \n", "3 NaN Cash N \n", "4 NaN Cash N \n", "\n", " debt_settlement_flag_date settlement_status settlement_date \n", "0 NaN NaN NaN \n", "1 NaN NaN NaN \n", "2 NaN NaN NaN \n", "3 NaN NaN NaN \n", "4 NaN NaN NaN \n", "\n", "[5 rows x 36 columns]" ] }, "execution_count": 42, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# only select 'object' dtype columns, which are usually strings\n", "loans.loc[:, loans.dtypes == 'object'].head()" ] }, { "cell_type": "code", "execution_count": 43, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Index(['id', 'member_id', 'funded_amnt_inv', 'int_rate', 'installment',\n", " 'annual_inc', 'url', 'dti', 'inq_last_6mths', 'mths_since_last_delinq',\n", " 'mths_since_last_record', 'revol_util', 'out_prncp', 'out_prncp_inv',\n", " 'total_pymnt', 'total_pymnt_inv', 'total_rec_prncp', 'total_rec_int',\n", " 'total_rec_late_fee', 'recoveries', 'collection_recovery_fee',\n", " 'last_pymnt_amnt', 'mths_since_last_major_derog', 'annual_inc_joint',\n", " 'dti_joint', 'open_acc_6m', 'open_act_il', 'open_il_12m', 'open_il_24m',\n", " 'mths_since_rcnt_il', 'total_bal_il', 'il_util', 'open_rv_12m',\n", " 'open_rv_24m', 'max_bal_bc', 'all_util', 'inq_fi', 'total_cu_tl',\n", " 'inq_last_12m', 'avg_cur_bal', 'bc_open_to_buy', 'bc_util',\n", " 'mo_sin_old_il_acct', 'mths_since_recent_bc',\n", " 'mths_since_recent_bc_dlq', 'mths_since_recent_inq',\n", " 'mths_since_recent_revol_delinq', 'num_tl_120dpd_2m', 'pct_tl_nvr_dlq',\n", " 'percent_bc_gt_75', 'revol_bal_joint', 'sec_app_inq_last_6mths',\n", " 'sec_app_mort_acc', 'sec_app_open_acc', 'sec_app_revol_util',\n", " 'sec_app_open_act_il', 'sec_app_num_rev_accts',\n", " 'sec_app_chargeoff_within_12_mths',\n", " 'sec_app_collections_12_mths_ex_med',\n", " 'sec_app_mths_since_last_major_derog', 'deferral_term',\n", " 'hardship_amount', 'hardship_length', 'hardship_dpd',\n", " 'orig_projected_additional_accrued_interest',\n", " 'hardship_payoff_balance_amount', 'hardship_last_payment_amount',\n", " 'settlement_amount', 'settlement_percentage', 'settlement_term'],\n", " dtype='object')" ] }, "execution_count": 43, "metadata": {}, "output_type": "execute_result" } ], "source": [ "loans.loc[:, loans.dtypes == 'float64'].columns" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Some of these are categorical, some of these are dates (which we might choose to drop or parse as dates), but some of these might even have continuous aspects, such as the employment length:" ] }, { "cell_type": "code", "execution_count": 44, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "10+ years 333964\n", "2 years 90476\n", "< 1 year 81758\n", "3 years 81013\n", "1 year 66927\n", "5 years 61623\n", "4 years 61101\n", "6 years 43380\n", "8 years 38349\n", "7 years 34708\n", "9 years 32224\n", "Name: emp_length, dtype: int64" ] }, "execution_count": 44, "metadata": {}, "output_type": "execute_result" } ], "source": [ "loans.emp_length.value_counts()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If you feel that dtypes are a good way to select features on your dataset, you can do so in ``ColumnTransformers`` even more easily with ``make_column_selector``:" ] }, { "cell_type": "code", "execution_count": 45, "metadata": {}, "outputs": [], "source": [ "from sklearn.compose import make_column_selector\n", "ct = make_column_transformer((OneHotEncoder(handle_unknown='ignore', sparse=False),\n", " # pass object dtype columns to OneHotEncoder\n", " make_column_selector(dtype_include='object')),\n", " # pass float and int dtypes to StandardScaler\n", " (StandardScaler(), make_column_selector(dtype_include=['float', 'int'])))" ] }, { "cell_type": "code", "execution_count": 46, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "loan_amnt int64\n", "int_rate float64\n", "home_ownership object\n", "open_acc int64\n", "grade object\n", "dtype: object" ] }, "execution_count": 46, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# we're only using some of the columns as the whole dataset is quite messy\n", "X = some_loans[['loan_amnt', 'int_rate', 'home_ownership', 'open_acc', 'grade']]\n", "X.dtypes" ] }, { "cell_type": "code", "execution_count": 47, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[('onehotencoder',\n", " OneHotEncoder(handle_unknown='ignore', sparse=False),\n", " ['home_ownership', 'grade']),\n", " ('standardscaler', StandardScaler(), ['int_rate']),\n", " ('remainder', 'drop', [0, 3])]" ] }, "execution_count": 47, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ct.fit(X)\n", "# we can look at the fitted transformers to see which columns where passed where\n", "ct.transformers_" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The benefit of using ``make_column_selector`` instead of passing a boolean mask is that the correct columns are determined when fitting the ``ColumnTransformer`` which avoids mistakes when creating the mask.\n", "The ``make_column_selector`` function can also match column names or exclude certain dtypes." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Collinearity & Redundancy\n", "```{sidebar} One-hot vs dummy naming\n", "In some communities, one-hot encoding refers to using one column per category, while dummy encoding refers to leaving out one of them. Using this nomenclature, ``pd.get_dummies`` does one-hot encoding by default.\n", "In pandas and scikit-learn the terms are used interchangeably and we're using them interchangeably in this book. TODO is this a good idea?\n", "```\n", "Coming back to one-hot encoding, there's one more important thing to know, which is that one-hot encoding is redundant. We can drop one of the columns for each categorical variable and still retain all the information: if all of the other columns are zero, the dropped one was 1, and if one of them is 1, the dropped one was 0. In other words, you could re-compute a column as one minus the sum of the other columns. Such a relationship is known as collinearity (or linear dependency). For some statistical models, the presence of such a relationship between the features can lead to numerical and statistical issues. Therefore, it is common in statistics to drop one of the columns. You can do that in pandas by using ``pd.get_dummies(X, drop_first=True)`` which will drop the first category for each feature:" ] }, { "cell_type": "code", "execution_count": 48, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
loan_amntint_rateopen_acchome_ownership_ANYhome_ownership_MORTGAGEhome_ownership_OWNhome_ownership_RENTgrade_Agrade_Bgrade_Cgrade_Dgrade_Egrade_Fgrade_G
080006.461201001000000
1600014.47801000010000
2100008.81701001000000
31000027.27400010000100
43500016.142201000010000
\n", "
" ], "text/plain": [ " loan_amnt int_rate open_acc home_ownership_ANY home_ownership_MORTGAGE \\\n", "0 8000 6.46 12 0 1 \n", "1 6000 14.47 8 0 1 \n", "2 10000 8.81 7 0 1 \n", "3 10000 27.27 4 0 0 \n", "4 35000 16.14 22 0 1 \n", "\n", " home_ownership_OWN home_ownership_RENT grade_A grade_B grade_C \\\n", "0 0 0 1 0 0 \n", "1 0 0 0 0 1 \n", "2 0 0 1 0 0 \n", "3 0 1 0 0 0 \n", "4 0 0 0 0 1 \n", "\n", " grade_D grade_E grade_F grade_G \n", "0 0 0 0 0 \n", "1 0 0 0 0 \n", "2 0 0 0 0 \n", "3 0 1 0 0 \n", "4 0 0 0 0 " ] }, "execution_count": 48, "metadata": {}, "output_type": "execute_result" } ], "source": [ "pd.get_dummies(X).head()" ] }, { "cell_type": "code", "execution_count": 49, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
loan_amntint_rateopen_acchome_ownership_MORTGAGEhome_ownership_OWNhome_ownership_RENTgrade_Bgrade_Cgrade_Dgrade_Egrade_Fgrade_G
080006.4612100000000
1600014.478100010000
2100008.817100000000
31000027.274001000100
43500016.1422100010000
\n", "
" ], "text/plain": [ " loan_amnt int_rate open_acc home_ownership_MORTGAGE home_ownership_OWN \\\n", "0 8000 6.46 12 1 0 \n", "1 6000 14.47 8 1 0 \n", "2 10000 8.81 7 1 0 \n", "3 10000 27.27 4 0 0 \n", "4 35000 16.14 22 1 0 \n", "\n", " home_ownership_RENT grade_B grade_C grade_D grade_E grade_F grade_G \n", "0 0 0 0 0 0 0 0 \n", "1 0 0 1 0 0 0 0 \n", "2 0 0 0 0 0 0 0 \n", "3 1 0 0 0 1 0 0 \n", "4 0 0 1 0 0 0 0 " ] }, "execution_count": 49, "metadata": {}, "output_type": "execute_result" } ], "source": [ "pd.get_dummies(X, drop_first=True).head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In ``OneHotEncoder``, you can specify ``drop='first'`` if you want to drop the first category, or you can specify the category you want to drop for each of the features.\n", "The only model in scikit-learn for which retaining all of the feature is an issue is ``LinearRegression`` (which we'll discuss in Chapter TODO), but otherwise you don't have to worry about it. For many of the model in sklearn that we'll see in Chapter TODO, it can matter which feature you drop, and it might be harder to interpret the model once you do. Therefore I usually recommend keeping all features in one-hot-encoding, even though the representation is redundant." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Dropping redundant binary features\n", "There is one exception though, which is binary features. If you have a binary categorical feature, i.e. a feature with two categories, it is obviously redundant to encode it using OneHotEncoder. Let's look at the 'term' variable which has two values, '36 months' and '60 months':" ] }, { "cell_type": "code", "execution_count": 50, "metadata": {}, "outputs": [ { "data": { "text/plain": [ " 36 months 715939\n", " 60 months 284061\n", "Name: term, dtype: int64" ] }, "execution_count": 50, "metadata": {}, "output_type": "execute_result" } ], "source": [ "loans['term'].value_counts()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If we encode this feature using OneHotEncoder, we will get two new features, one corresponding to the 36s month term, and the other to the 50 months term:" ] }, { "cell_type": "code", "execution_count": 51, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array(['term_ 36 months', 'term_ 60 months', 'grade_A', 'grade_B',\n", " 'grade_C', 'grade_D', 'grade_E', 'grade_F', 'grade_G'],\n", " dtype=object)" ] }, "execution_count": 51, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ohe = OneHotEncoder().fit(loans[['term', 'grade']])\n", "# get feature names, passing in the original feature names\n", "ohe.get_feature_names(['term', 'grade'])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "However, these two features encode exactly the same information, in that the 36 months term feature is just 1 minus the 60 month term feature. This redundancy is not useful and will make any model less easy to interpret, and we can drop it using ``drop='if_binary'``:" ] }, { "cell_type": "code", "execution_count": 52, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array(['term_ 60 months', 'grade_A', 'grade_B', 'grade_C', 'grade_D',\n", " 'grade_E', 'grade_F', 'grade_G'], dtype=object)" ] }, "execution_count": 52, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ohe = OneHotEncoder(drop='if_binary').fit(loans[['term', 'grade']])\n", "ohe.get_feature_names(['term', 'grade'])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now there's only one feature to represent this information, while the 'grade' feature with seven categories has retained all columns." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Impact Encoding & Dealing with many categories\n", "Dealing with categorical features with many categories can be a bit tricky, as directly applying one-hot encoding is often not effective.\n", "Let's look at the ``emp_title`` feature from the lending club data, which provides a job description of the borrower:" ] }, { "cell_type": "code", "execution_count": 53, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(383181, 145)" ] }, "execution_count": 53, "metadata": {}, "output_type": "execute_result" } ], "source": [ "loans_paid.shape" ] }, { "cell_type": "code", "execution_count": 54, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Teacher 6819\n", "Manager 6609\n", "Owner 3977\n", "Driver 2982\n", "Registered Nurse 2859\n", " ... \n", "Corporate Delivery Specialist 1\n", "telecom 1\n", "Sewer Manitenance supervisor 1 1\n", "Regulatory Compliance Survey Manager 1\n", "Lead payroll admin 1\n", "Name: emp_title, Length: 108751, dtype: int64" ] }, "execution_count": 54, "metadata": {}, "output_type": "execute_result" } ], "source": [ "title_counts = loans_paid.emp_title.value_counts()\n", "title_counts" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We loaded 1,000,000 rows of data, and it contains 224,610 different employment titles. Several of the titles are very common, such as Teacher and Manager, while some of the titles appear only once, like 'Court Evaluator'. TODO numbers\n", "Let's see how many of the value appear only once:" ] }, { "cell_type": "code", "execution_count": 55, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "85905" ] }, "execution_count": 55, "metadata": {}, "output_type": "execute_result" } ], "source": [ "title_once = title_counts == 1\n", "title_once.sum()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "That's most of them! Clearly it will be hard to learn from a category for which we have only seen one example. This is likely another example of a power-law distribution: most of the job titles appear only once, there are some that appear several times, but then there are some that appear much more frequently than all the rest, such as Teacher and Manager. We can validate this by looking at the counts of the counts:" ] }, { "cell_type": "code", "execution_count": 56, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 56, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAD8CAYAAABw1c+bAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3deXhV1b3/8fc3MxkhA4iEGEIYRREMg7YqDlUcsbVVqNahKNU69N7e3oqt99r52l9v79Nr1SoqQqvCRWstKtZWW6UqMyozEkAgDCEQIBNkXL8/EjDQpJyTc072GT6v5+F5ctbZe58vi8N373z32muZcw4REYkdcV4HICIi3UuJX0Qkxijxi4jEGCV+EZEYo8QvIhJjlPhFRGJMgtcBAOTm5rrCwkKvwxARiSgrVqzY55zL83e/sEj8hYWFLF++3OswREQiiplt68p+KvWIiMSYoCd+M5tgZn83syfMbEKwjy8iIoHxKfGb2Uwz22tma05on2hmG82s1MymtzU7oAZIAcqCG66IiATK1yv+WcDE9g1mFg88BlwODAemmNlw4O/OucuB+4EfBi9UEREJBp8Sv3NuIVB5QvNYoNQ5t8U51wDMBSY551ra3j8AJActUhERCYpARvX0A3a0e10GjDOzLwGXAT2BRzvb2cymAdMACgoKAghDRET8EcjNXeugzTnnXnbOfcM5d4Nz7p3OdnbOzXDOlTjnSvLy/B6GCsChw40sWL27S/uKiMSqQBJ/GdC/3et8YJc/BzCzq81sxqFDh7oUwGN/K+WeF1by8Y6DXdpfRCQWBZL4lwGDzGyAmSUBk4H5/hzAOfeqc25aVlZWlwK456Ji8jKSuf/3q2hsbjn5DiIi4vNwzjnAImCImZWZ2VTnXBNwD/AmsB6Y55xb68+HB3rFn5mSyI8njWDDnmqefHdzl44hIhJrLByWXiwpKXGBTNlw9/Mr+cu6chZ86zyKe6cHMTIRkfBlZiuccyX+7ufplA2BXvEf9YNrTqdHUjwPvLyKlhbvT2QiIuHM08QfaI3/qLyMZB68chjLPj3A80u6NGeRiEjMiJpJ2r58dj7nDcrl4Tc2sOvgYa/DEREJW1FR6mk7Fj/74hm0OHjwlTWEw70LEZFwFBWlnqP6Z6fyb5cO5q8b9jL/Y78eKRARiRlRU+o56rbPDWBkfhY/fHUdlbUNXocjIhJ2oqbUc1R8nPHwdWdSdbiRn7y2LmjHFRGJFlFV6jlqWN9M7powkJc/3Mk7G/cG9dgiIpEu6ko9R91zUTED89L4/h/WUFvf5HU4IiJhI2oTf3JCPD+/7kx2HTrML97c6HU4IiJhI+pq/O2VFGbztfGnMXvRp6zYdiAknyEiEmmissbf3ncnDqVvZgrTf7+K+qbmkH2OiEikiNpSz1HpyQn89ItnsGlvDY//TTN4iohEfeIHuHBobyaddSqPv1PKJ+XVXocjIuKpmEj8AP951XDSkxP47kuraNYMniISw6L65m57OenJPHT16Xy04yCzP/g05J8nIhKuov7mbnuTzjqVCUPy+MWbG9lRWdctnykiEm5iptQDrTN4/uTaEZjB9zWDp4jEqJhK/AD5vVL57mVDWPhJBX/4cKfX4YiIdLuYS/wAXzunkNEFPfnRa+vYV1PvdTgiIt0qJhN/fJzx8+vOpK6+mR++qhk8RSS2xGTiBxjUJ4O7Lyzm1Y938fb6cq/DERHpNjEznLMjd00YyJA+GTz4yhqqjzR6EoOISHeLqeGcJ0pKiOPh685gT9URfv6nDZ7EICLS3WK21HPUqIJe3HbuAJ5bvJ2lWyu9DkdEJORiPvEDfOeyweT36sH036/iSKNm8BSR6KbED6QmJfCzL57Bln21/Pqvm7wOR0QkpJT425w/OI/rRufz5LtbWLeryutwRERCRom/nQevHEZWj0Smv7yKpuYWr8MREQkJJf52eqUl8YNrTmdV2SGeff9Tr8MREQmJkCR+M0szsxVmdlUojh9KV53Zl0uG9eaXf9nItv21XocjIhJ0PiV+M5tpZnvNbM0J7RPNbKOZlZrZ9HZv3Q/MC2ag3cXM+PG1I0iIi+OBl1drBk8RiTq+XvHPAia2bzCzeOAx4HJgODDFzIab2SXAOiBi50Hom9WD6ZcP5YPN+3lxeZnX4YiIBFWCLxs55xaaWeEJzWOBUufcFgAzmwtMAtKBNFpPBofNbIFzLuLulH51bAHzP97Fj19bR0lhL4ry0r0OSUQkKAKp8fcDdrR7XQb0c8593zn3L8ALwFOdJX0zm2Zmy81seUVFRQBhhEZcnPE/148kId6487kV1NY3eR2SiEhQBJL4rYO2YwVx59ws59xrne3snJvhnCtxzpXk5eUFEEbo5PdK5ZEpoyjdW8P9v1+ler+IRIVAEn8Z0L/d63xglz8H8Hp2Tl+cNyiP71w2hNdW7eaZ97Z6HY6ISMACSfzLgEFmNsDMkoDJwHx/DuD17Jy+uuuCgVx2eh/+640NLNq83+twREQC4utwzjnAImCImZWZ2VTnXBNwD/AmsB6Y55xb68+HR8IVP7QO8fzvr4ykMCeVe+esZPehw16HJCLSZRYOdeuSkhK3fPlyr8M4qdK91Ux69H0G9cng/74xnuSEeK9DEpEYZmYrnHMl/u4X0ytw+au4dwb//ZWRfLTjID/SWr0iEqFiegWurrj8jL7cecFAnl+ynXnLd5x8BxGRMKNJ2rrgO5cO5nPFOTz4yhpWl0XGbysiIkep1NMFCfFxPDJ5FHnpydz53Aoqaxu8DklExGcq9XRRTnoyv7lpNBU19dw350OaW7y/SS4i4guVegJwZn5PfjJpBO+V7uOXf97odTgiIj5RqSdA14/pz5SxBTz+zmb+tGaP1+GIiJyUSj1B8INrhjOyf0++8+LHlO6t8TocEZF/SqWeIEhOiOeJm0aTnBDHnc+toEYzeYpIGFPiD5K+WT349VdHsaWihn9/8WPN5CkiYUs1/iA6d2AuD1w+jDfW7OHJhVu8DkdEpEOq8QfZ7ecN4Koz+/LwGxv4w4datlFEwo9PSy+K747O5Lm/poHvvLiK9OREvjC8j9dhiYgcoxp/CKQkxvPULSWM6JfF3S+s5IPSfV6HJCJyjBJ/iKQnJzD7tjEMyEnj9t8u58PtB7wOSUQE0M3dkOqZmsTvpo4lNz2ZW59dxsY91V6HJCKim7uh1jszhedvH0dKYhw3PbOEbftrvQ5JRGKcSj3doH92Ks9NHUdTcws3Pr2EPYeOeB2SiMQwJf5uMqhPBrO/PpaDdY3c9MwSTeUsIp5R4u9GZ+b35KmbS9heWcctM5dSfaTR65BEJAYp8Xezcwbm8JsbR7N+dxVTZy/nSGOz1yGJSIxR4vfAxcP68MvrR7Ls00q++fxKGptbvA5JRGKIhnN6ZNJZ/fjRpBH8dcNevvfyak3qJiLdRsM5PfS18adx38WDeHFFGb94Uyt4iUj30Fw9HvvXSwZRUV3P4+9sJi8jmds+N8DrkEQkyinxe8zM+Mm1I9hfU8+PXltHbnoyV4881euwRCSK6eZuGIiPMx6ZMooxp2Xz7Xkf8b4mdROREFLiDxMpifE8dXMJRbnpfON3K1izM/ZueItI91DiDyNZqYnM/vpYsnokcuuzyzSvj4iEhBJ/mDklK4XZXx9LU0sLN89cSkV1vdchiUiUCXriN7NhZvaEmb1kZncF+/ixoLh3OjNvHcPeqnpufHqxkr+IBJVPid/MZprZXjNbc0L7RDPbaGalZjYdwDm33jl3J3A9UBL8kGPD6IJePHNrCTsqDzPlqcXsrdaMniISHL5e8c8CJrZvMLN44DHgcmA4MMXMhre9dw3wHvB20CKNQecOzOXZ28aw6+BhJs9YTHmVkr+IBM6nxO+cWwhUntA8Fih1zm1xzjUAc4FJbdvPd86dC9wYzGBj0fiiHGZ/fSzlh44wecZizeUvIgELpMbfD9jR7nUZ0M/MJpjZI2b2JLCgs53NbJqZLTez5RUVFQGEEf3GFGbz26ljqaiu54YZi9h18LDXIYlIBAsk8VsHbc45945z7j7n3Decc491trNzboZzrsQ5V5KXlxdAGLHh7NNak39lTQM3zFhE2YE6r0MSkQgVSOIvA/q3e50P7PLnALE8O2dXjC7oxe9uH8fBukYmz1isK38R6ZJAEv8yYJCZDTCzJGAyMN+fA8T67JxdcVb/njw3dRyH6hq58ekl7NUNXxHxk6/DOecAi4AhZlZmZlOdc03APcCbwHpgnnNurT8friv+rhnZvyfP3jaG8qoj3Pj0EvbXaJy/iPjOwmEBkJKSErd8+XKvw4g4izbv59Znl1KUl86cO8bRMzXJ65BEpBuZ2QrnnN/PS2kFrgh2zsAcnrq5hM17a7h55lKqtHi7iPhAK3BFuPMH5/Gbm0azblcVtz27jMMNWrxdRP45TdIWBS4e1odHpoxi5fYD/Ov/fURLi/flOxEJXyr1RIkrzujLg1cO509r9/DzP23wOhwRCWMq9USRr3+ukK+NP40nF27hhSXbvQ5HRMKU1tyNImbGQ1cPZ8eBOv7jj2vI79WD8wfrqWgROZ5KPVEmIT6OR786mkG90/nm8yvZuKfa65BEJMyo1BOF0pMTmHnrGFKT4vnaM0vYUlHjdUgiEkY0qidKndqzB8/dPo7mFseUpxazdZ/W7xWRVkr8UWxwnwxeuGM8Tc2OyTMWKfmLCKAaf9Qbckpr8m9sdkyZsZhPlfxFYp5q/DGgNfmPo6G5hS8/sYhVZQe9DklEPKRST4wYekom874xnuSEOG54cjFvrSv3OiQR8YgSfwwp7p3BH+4+l0F90pn2u+XMen+r1yGJiAeU+GNM74wU5k4bz8XD+vCDV9dxzwsrqajWfP4isUQ3d2NQalICT9x0Nt/+wmD+vLaci3/5DnOWbtfkbiIxQguxxLjNFTV87+XVLNlayVfOzucXXxnpdUgi4qOIXIhFvDcwL52508Yz7fwiXlxRxgeb93kdkoiEmBK/YGZ8+wuD6Z/dg//841oam1u8DklEQkiJXwBISYznB1efTuneGp7VaB+RqKbEL8dcPKwPlwzrza/e2sTuQ4e9DkdEQkSJX47z0NWnt07sNmMxTy3cwr4aDfUUiTYazinH6Z+dym9uGk2vtCR+umA943/2Nj96dR219U1ehyYiQaLhnNKpT8qrefb9rcxZuoN+PXvwsy+dwQVa0UskbGg4pwTd4D4Z/NeXzuTFO88hJTGOW59dyrzlO7wOS0QCpMQvJzWmMJvX7zuPzxfncv/vVzFvmZK/SCRT4hefpCTG89TNJZw3KI/7X17FXzdodk+RSKXELz5LSYxnxtfOpig3jZ++vp4mPeglEpGU+MUvKYnx/PtlQ9lcUctLK8q8DkdEukCJX/x22el9GFXQk1+9tYkjjc1ehyMiflLiF7+ZGfdPHMqeqiPc+dwKXvlwp8b5i0SQkCR+M7vWzJ4ysz+a2aWh+Azx1viiHO67qJhVZYf4l//7iNtmLSMcngkRkZPzOfGb2Uwz22tma05on2hmG82s1MymAzjnXnHO3QHcCtwQ1IglbHz70iEs//4lPHjlMJZureTt9Xu9DklEfODPFf8sYGL7BjOLBx4DLgeGA1PMbHi7TR5se1+iVFycccu5hQzITeMXb26kWat4iYS9BF83dM4tNLPCE5rHAqXOuS0AZjYXmGRm64GHgTeccys7Op6ZTQOmARQUFPgfuYSNxPg4/u3Swdzzwof8YP5aRvTLJKtHEj1TE8lISaAgO5WMlESvwxSRNj4n/k70A9o/xlkGjAPuBS4Bssys2Dn3xIk7OudmADOgda6eAOMQj10xoi/nDy7jd4u3/cN7GckJrP7hZR5EJSIdCTTxWwdtzjn3CPDISXc2uxq4uri4OMAwxGtxccbs28ZwuLGZ4f/55nHvXTXyVJqaW0iI1yAykXAQ6P/EMqB/u9f5wC5fd3bOveqcm5aVlRVgGBIOzIzUpAR+f9e5XHvWqXzl7HyyeiQyZ+l2xv3sbR764xrW7NQU3CJe82ta5rYa/2vOuRFtrxOAT4CLgZ3AMuCrzrm1Ph7v6BX/HZs2bfIvcokI9U3NvLuxgj9+tIu/rC+nsbmF56aO43PFuV6HJhLxujots8+J38zmABOAXKAceMg594yZXQH8CogHZjrnfupvEJqPPzYcqmvki4+/T31TC3/6l/N0w1ckQF1N/P6M6pnSSfsCYIG/Hwyq8ceSpuYWPi47SO/MZBZvqeR3i7fxzQn6dxfxQqA3dwPinHsVeLWkpOQOL+OQ0Lvx6SUs2VpJRnIC15fkc83IU70OSSRmaZiFdIsvje5HckIc8fHGBYN7k98r1euQRGKWFluXbnHDmAJev+88CrJTufuFlXxr7occqmv0OiyRmKTF1qVbNTa38PjfNvPrv24iJz2J//flkVrAXaSLtNi6RITE+Di+dckgfnLtCMqr6rll5lLeXq9lHEW6k6c3dzWqJ3YV5aUf+3nq7OWM7N+Ti4b05qKhvTn91Ezi4jp6KFxEgkGlHvFMS4tj3e4q/rphL88t3sbe6vpj7y164CL6ZvXwMDqR8BfycfwiwRYXZ4zol8WIflmkJMbxswUbABjeN5PUJH01RUJF/7vEc2t3HWJ/TQMAvTOSee3ez6vUIxJCqvGL5x5+YwN/37QPgIF56bQ4R1yHE7+KSDB4OqpHs3MKwKNTRnPvRcWkJyewaMt+vjX3IzbuqfY6LJGopZu7EjYO1jXwzHtbefb9T6mpb2J430zGFWUzviiHcQOy6Zma5HWIImEl5LNzhpISv7R3oLaBF5Zu5/3SfazYdoD6phYAhp6SwfiiHMYXZTN2QA7ZaToRSGxT4peotKOyjjlLt/P66t1s21933Hsj+/fkualjNb2zxKyIHM6pm7vyz6zZeYjrn1xEXUPzsbbUpHhy05PJTU9iQG46CXF6+FzEX5qWWcLWLTOXHpf0v3/FMO44v8jDiESigy6XJGx974ph3Hpu4bHXr67axZaKGu8CEokSqvFL2Nt96DBPvLOZuct20NDcQkdf2emXD+XOCwZ2f3AiHtLsnBK1+mb14IeTRvD+9Iv49iWDO9xmZH7Pbo5KJHIp8UvEyE1P5t6LB/G/k886rv3S4X1obnGEw2+vIpFAc/VIxJl0Vj8mndWP7fvreH7pNuYt28Gf15VTlJvGjeNP48uj88lK1RBPkc54WuNvN5zzjk2bNnkWh0SG22cv4631e0+6XUpiHDNvGcO5xbndEJWIdyKyxq+5esQfg/tk+LTdkcYW0lP0y6xIZzSqRyJSQ1MLOw8eZntlHdv317LjwGFq65t4Y80eKmsbyE5L4q1vX6BpHSSqReSTuyJdlZQQx4DcNAbkpgF5HKht4Kpfv0dlbQOZKQlcdWZf0pLjvQ5TJCxpVI9EhbTkBPIykgG4ZHgf7rt4EMkJSvwiHdEVv0SFpIQ45n3jHH755408/d5W/ry2nGtHnUpKQjyVdQ0MyEnjmxcWE6+VvUSU+CV6JCXE8cAVw/hKSX8efmMDf1i5k9p2c/0U5aXTs5Nhns5BedUR4uOMa0aeqqUfJaop8UvUKe6dztO3tN7v+tVbn/Crt1qHCt/9wkqf9s/skcBFQ/uELD4RrynxS0S7+/mVvL56d0DHyEhOoDA3jcLcNMYOyObCIb2DFJ1IeAp64jezIuD7QJZz7svBPr5Ie+OKsv1O/NeNzueei4rp36sHCfEa3yCxx6dx/GY2E7gK2OucG9GufSLwv0A88LRz7uF2773ka+LXOH4JpYrqehas3s1D89cea/vwP75AL43xlwgX6nH8s4BHgd+2+8B44DHgC0AZsMzM5jvn1vkbhEgoHDrcyH1zPuTvmypoaXd9k5uexL+/9LFP0z8cdf/Eodw1QdM+S3TwKfE75xaaWeEJzWOBUufcFgAzmwtMAnxK/GY2DZgGUFBQ4GO4Ir5rbnE0NLUwrG/mP7y38+ARv45VXuXf9iLhLJACZz9gR7vXZUA/M8sxsyeAUWb2QGc7O+dmOOdKnHMleXl5AYQh0rHstCTmTBvP0FMyqT7SdNyf2vomCrJTye/VgyQf6vx/WVfOxj3V3RC1SOgFcnO3o4HOzjm3H7jTpwNosXXpBsP6ZlBaUcP63VU0NLX8w/uJ8caIfpkU5aZ3+oBXSmJcp88AiESaQBJ/GdC/3et8YJc/B9Bi69Idbj+viNvPK6KxuYXSvTWs3nmItTsPsXrnIdbtruJIYwtrdlaxeW8tp+Wk0jM1kdEFvTj7tF6MKuilid4k6gSS+JcBg8xsALATmAx81Z8D6IpfulNifBzD+ma21vxLWq9Zmppb2LKvljVtJ4KyA4cprzrCjIVbaGq7I1yUl8bZbSeCs0/rxcC8dD3ZKxHN1+Gcc4AJQC5QDjzknHvGzK4AfkXrcM6ZzrmfdiUIDeeU7rLwkwo+3H7wpNsdbmxm9c6DfLT94HHTPgBkpiQwqqAXFw7J4+ZzCnUSEM+EdDinc25KJ+0LgAX+fuhRuuKX7vbbRdt4a315QMeoOtLEu59UsHZXFZPHFpASp1lAJbJoIRaJKc45qg43sa+2nsraBvbX1LOvpuHYz/trG9hf08D+tvcraxuOewagvV6pieSkJ5OTlkRuejLZaUnkpCcda8tJa/05Nz2JzJRE/WYgQaeFWEROUDj9db/3yctIJi89mYS4OHpnpJCbnszBukYqaxtoaD5+RNCBukYO1DVS6sNx4+OM7LQk8nv14PEbR9M3q4ffsYkEi6eJX6UeCaU+mcmUV9X7tU9FdT0V1fUM6ZNB/+xUAPJ7BS+m9OR4n54bEAkllXpE2ny6r5YnF27mpRVltDi4+sy+3DWhmCGn+LbIu0h3U6lHxA8rtlXy3OLtHb5Xclo2i7bs55WPdvHKR349mhIUr937eUb0y+r2z5XYoVKPxKSyA4d5r3Qf+2vqO71565WyA4eV+CWkVOqRmNbc4jhQ18C+mvpj9f32P1fU1LOvuoGKmtZRPh3JSE4gt+2mcF5G6yievIyjPycf+zknLZmkBNX3JXhU6hHpgvg4Ize9NUEPPeWfb9vY3EJlbcNxJ4UTTxbr91Sxr7qeqiNNHR6jZ2oieenHnxCOO0GkJ5ObkUROWrIWhpeQUeKXmNaVIZ+BOFjXyMG6Rjbtrfmn28UZZKe1/vZwzVmn8s0JKodK8KjGL9INxhdld3nfBF35S5Cpxi8SIm+vL2fq7OW8cPs4zi3O9TociUJdrfHrTpNIiLxfup/khDhGnxbEJ8BEgkCJXyRE3i/dx9gB2aQkahI3CS+6uSsxb87S7Tzw8uqQHHtjeXW330Bub8n3LqZPZopnny/hydMrfjO72sxmHDp0yMswJMbp1qnEGt3cFQnQ4YZmNpZXs353Fet3V7FuVxUb9lRTU986lj/OoDA3jWF9MxneN5NhfTMY1jeTUzJTMNNpR7pOD3CJeKRHUjxn9e/JWf17HmtraXGUHTjMuraTwfrdVawqO8jrq3Yf26ZXauKxpSCHtZ0QBvXO0NO9EnJK/CIhEBdnFOSkUpCTysQRnz0SXHWkkY17qlm367MTwvNLtnGksXWu/4Q4o7h3ettvBp+dEHLSk736q0gUUuIX6UaZKYmMKcxmTOFnD3Q1tzi27qs9diJYv7uK9zfv4+UPdx7bpndGcmup6NTMtpJRBgNy0zWtg3SJEr+Ix+LbrvKLe6dz9chTj7VX1jZ8dt9gdxXrd1fzwd+30Njcel8uOSGOIadkMOyUz04IQ/tmkJmS6NVfRSKEEr+ID77z4se8tKLM6zCOU9/UwqqyQ6wq839U3KcPXxmCiCRSaDiniA+276/zOgSRoNFwThE/VR9p5Pkl23nmva1UVNczMj+LuyYM5NLhpxCnmrt0o64O51TiF+miI43NvLxyJ08u3My2/XUU5aVx5/kDuXZUPw3JlG6hxC/ikeYWx4LVu/nNO5tZt7uKUzJTuP28AUwZW0Basm6jSego8Yt4zDnHwk37ePxvpSzZWknP1ERuOaeQW84tJDstyevwJAop8YuEkRXbDvDEu5v5y7pyeiTGM3lsf24/r4h+PXt4HZpEESV+kTC0qbya37y7mfkf7QLg2lH9uPOCIop7Z3gcmUQDJX6RMFZ2oI6n/76Vucu2U9/UwqXD+3DXhOLj5vcR8ZcSv0gE2F9Tz+wPPmXWB59SdaSJc4pyuGvCQM4blKuZOsVvYZP4zSwNeBxoAN5xzj1/sn2U+CXW1NQ3MWfJdp5+bwvlVfWM6JfJXRcUM3HEKZp/R3wW0jV3zWymme01szUntE80s41mVmpm09uavwS85Jy7A7jG34BEYkF6cgJ3nF/Ewu9eyM+vO4Pa+mbufmEll/zPu8xdup36pmavQ5Qo5usg41nAo8BvjzaYWTzwGPAFoAxYZmbzgXzg6Dp2+vZKVDvrR3/mYF1j0I63dV8t019ezfQQLQUp4ee1ez/PiH5Z3fqZPl3xO+cWApUnNI8FSp1zW5xzDcBcYBKtJ4H8kx3fzKaZ2XIzW15RUeF/5CJhIJhJX2JT2YHunwcqkMcK+wE72r0uA8YBjwCPmtmVwKud7eycmwHMgNYafwBxiHhGs1xKJAok8Xd0B8o552qB23w6gNnVwNXFxcUBhCEiIv4IZCapMqB/u9f5wC5/DuCce9U5Ny0rq3vrWyIisSyQxL8MGGRmA8wsCZgMzPfnAJqPX0Sk+/k6nHMOsAgYYmZlZjbVOdcE3AO8CawH5jnn1vrz4briFxHpfj7V+J1zUzppXwAs6OqHq8YvItL9PF0tQlf8IiLdT8sEiYjEGC22LiISY8Jidk4zqwC2tb3MAk48E5zY1v51LrAvhOF1FE8w9zvZdp2970s/ddR24utI7r9g9l1H7dHcd75sF8rvnvrun7f7+t07zTmX50O8x3POhdUfYMbJ2tq/BpZ3dzzB3O9k23X2vi/9dLK+i/T+C2bf+dhXUdN3we4//b+NrO9eONb4O5rm4cS2TqeCCIGufpav+51su87e96WfOmrrzr4L5PN82S+YfddRezT3nS/bRfJ3L5L6rqP2kPZdWJR6AmFmy10X5qOWVuq/rlPfdZ36LjCB9l84XvH7a4bXAQ9zZnYAAAKhSURBVEQ49V/Xqe+6Tn0XmID6L+Kv+EVExD/RcMUvIiJ+UOIXEYkxSvwiIjEm6hK/maWZ2Wwze8rMbvQ6nkhiZkVm9oyZveR1LJHIzK5t+9790cwu9TqeSGJmw8zsCTN7yczu8jqeSNOW91aY2VW+bB8Rid/MZprZXjNbc0L7RDPbaGalZja9rflLwEvOuTuAa7o92DDjT9+51vWTp3oTaXjys/9eafve3Qrc4EG4YcXPvlvvnLsTuB6I+WGefuY8gPuBeb4ePyISPzALmNi+wczigceAy4HhwBQzG07rSmBH1wJu7sYYw9UsfO87+Uez8L//Hmx7P9bNwo++M7NrgPeAt7s3zLA0Cx/7zswuAdYB5b4ePCISv3NuIVB5QvNYoLTtKrUBmAtMonVJyPy2bSLi7xdKfvadnMCf/rNWPwfecM6t7O5Yw42/3z3n3Hzn3LlAzJdo/ey7C4HxwFeBO8zspHkvkMXWvdaPz67soTXhjwMeAR41syvp/kfsI0WHfWdmOcBPgVFm9oBz7r88iS78dfbduxe4BMgys2Ln3BNeBBfmOvvuTaC1TJtMAIs7RbkO+845dw+Amd0K7HPOtZzsQJGc+K2DNuecqwVu6+5gIkxnfbcfuLO7g4lAnfXfI7ReeEjnOuu7d4B3ujeUiNNh3x37wblZvh4okkshZUD/dq/zgV0exRJp1HeBUf91nfqu64LWd5Gc+JcBg8xsgJklAZOB+R7HFCnUd4FR/3Wd+q7rgtZ3EZH4zWwOsAgYYmZlZjbVOdcE3AO8CawH5jnn1noZZzhS3wVG/dd16ruuC3XfaZI2EZEYExFX/CIiEjxK/CIiMUaJX0Qkxijxi4jEGCV+EZEYo8QvIhJjlPhFRGKMEr+ISIxR4hcRiTH/HyHzYIqehxB8AAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "# a power-law distribution looks like a straight line in a log-log plot. This seems to hold reasonably well here.\n", "title_counts.value_counts().plot(logy=True, logx=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Using a one-hot encoding on 224,610 categories will produce a feature space that will likely make learning hard. Even if we restrict ourselves to the categories that appear at least twice, we would still have over 50,000 features.\n", "One option is to define a cut-off, and only look at the top k, say top 50 or top 100, most common values, and move everything else into an \"other\" category. Let's have a look at what the common values are:" ] }, { "cell_type": "code", "execution_count": 57, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Text(0.5, 1.0, '50 most common job titles')" ] }, "execution_count": 57, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAsgAAAJICAYAAAB8GxKuAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nOzdebhcVZ318e8yIBACQYSmA41GkUHGQBKUGRTpdgQUDYPNoM1gKzS2YqMoBlAQ8VUBRQ00AsokIIiAgELCPENGJhFCK6AyRsJMWO8fZ1dyUlTdIbkh995an+e5T6r2OWeffer6dP/uZtdesk1ERERERFTetLgHEBERERHRn6RAjoiIiIioSYEcEREREVGTAjkiIiIioiYFckRERERETQrkiIiIiIiaFMgREdHRJP1O0l49OG+kJEtaYiHuNUPStl0cnyTpPxa0/4joGymQIyL6UClwXpQ0u/zc13T8/ZLulfS8pImS3r64xlrGs7ek6xfnGBY32x+0fXpf9yvpNEnfarrXerYnlePjJf2yr+8bEQsvBXJERN/7gu1h5WftRqOklYBfA98AVgRuB85dTGOMiIg2UiBHRLxxPg7MsH2e7ReB8cBGktZpdXKZjf6WpBvLbPRvJb1V0pmS/iHpNkkja+dvXtpmlX83rx3bW9KDkp6V9JCkPSS9G/gpsFnp/5k241hR0s8lPSrpaUkX1Y7tK+kBSU9JuljSqrVjlvSfkv5Y7nuUpDUk3VTG/ytJby7nbivpL5K+Iunvkh6TtJOkD0m6v/T/tVrfS0n6YRnTo+X1Uk19fanW1z7tfin1ZQ2S3iTp65IeLteeIWl40yWfKfd8TNKX2vS5H7AH8JXG7660z5S0vaR/A74GjCvHp7Tp5zOS7imf+xWN/+Kgyg/KGGdJmipp/XbPGBG9kwI5IqLvHSPpCUk3NK03XQ+YWwjZfg74U2lvZ1fg34HVgDWAm4CfU81A3wN8E6oiFrgUOAF4K/B94NJSUC9b2j9oezlgc2Cy7XuAA4Cbymz3Cm3G8AtgaBnnPwE/KPd8H3AM8ClgBPAwcE7Ttf8GjAbeC3wFmEBVOK4OrA/sVjv3n4Gly7MeDpwMfLpcvxVwuKR3lnMPK32OAjYCNgW+3tTX8NLXZ4EfS3pLm+er27v8bAe8ExgG/KjpnO2ANYEdgEMlbd/cie0JwJnAd8tn+9Gm45cDRwPnluMbNfchaSeqIvrjwMrAdcDZ5fAOwNbAWsAKwDjgyR48X0T0QArkiIi+9T9UhdVqVMXgbyWtUY4NA2Y1nT8LWK6L/n5u+0+2ZwG/A/5k+w+2XwXOAzYu530Y+KPtX9h+1fbZwL1AozB7DVhf0jK2H7M9oycPI2kE8EHgANtP237F9jXl8B7AqbbvtP0S8FWq2eiRtS6Otf2Pcr/pwJW2H6w9z8a1c18Bvm37FapCeyXgeNvPlutnABvW7n2k7b/bfhw4guoPiXpfR5bxXgbMBtame3sA3y9jnF2eaVfN/8W8I2w/Z3sa1R8ru7XqqA/sDxxj+57y+z4aGFVmkV+h+t/NOoDKOY8tonFEdJwUyBERfcj2LaWge6l88esG4EPl8Gxg+aZLlgee7aLLv9Vev9Di/bDyelWqGdy6h4HVykz1OKrZ4sckXdpuWUcLqwNP2X66xbH57lkKyiep/jjo7fgBnrQ9p3as1fXtnvfh0lbv69Xa++eb7tVOq36XAFaptf25i/v2pbcDx0t6pix/eQoQ1e/0aqqZ7R8Df5M0QVLz/7YiYgGlQI6IWLRMVdRANQM69z+ll6UPa5T2hfUoVUFV9zbgEQDbV9j+ANVSiHupli80xteVPwMrSmq1/GK+e5bneWvjnotY8/O+rbQtin5fZf5CffUe3re7z7Ynn/3+tleo/Sxj+0YA2yfYHk219GUt4JBu+ouIHkqBHBHRRyStIOlfJS0taQlJe1CtE72inHIh1TKHT0hammqd7VTb9/bB7S8D1pK0e7n3OGBd4BJJq0j6WClgX6KayW7M1P4N+JfGl+Walf9s/zvgJElvkbSkpK3L4bOAfSSNKl+QOxq4xfbMPnie7pwNfF3Syqp2Bzkc6Ist084GvijpHZKGMW+dcH02+huShkpaD9iH9juR/I1quU07fwNGSmr3/4t/Cny13AdJwyV9srweK+k9kpYEngNeZN7vNCIWUgrkiIi+syTwLeBx4AngQGAn2/cBlLWynwC+DTwNvIfqS3gLzfaTwEeAL1Etc/gK8BHbT1D93/ovUc10PgVsA/xnufRqqhnsv0p6ok33/0615vVe4O/AweWeV1FtWXcB8BjVbHifPE8PfItqm7ypwDTgztK2sE6l+lLitcBDVIXngU3nXAM8AFwFfM/2lW36+l9g3bJE4qIWx88r/z4p6c7mg7YvBI4FzpH0D6o13B8sh5en+q8AT1Mt83gS+F6PnjAiuiW7u//CExERMXhJuhY4xfYZi3ssEdE/ZAY5IiI6lqShVMsgHlrcY4mI/iMFckREdCRJ/wT8lWrJREfHbUfE/LLEIiIiIiKiJjPIERERERE1S3R/SnSylVZaySNHjlzcw4iIiIjoU3fccccTtldudSwFcnRp5MiR3H777Yt7GBERERF9SlJz+uhcWWIREREREVEzIGeQJc2h2hh+Caqtef7d9jML0M+qwAm2d2lzfAVgd9snLcx4u7j/eGC27X67ufu0R2Yx8tBLF/cwemzmdz68uIcQERERA9xAnUF+wfYo2+tTpUJ9fkE6sf1ou+K4WIF5aVM9ospi/VwlDVmc94+IiIgYyAZqgVx3E7AagKQ1JF0u6Q5J10lap9Z+s6TbJB0paXZpHylpenm9nqRbJU2WNFXSmsB3gDVK23HlvENKP1MlHVHr5x5JJ1HFna7e6rxy7mGS7pP0B2DtVg8k6TRJJ0i6UdKDknYp7dtKuqR23o8k7V1ez5R0uKTrgU9KOkjS3eX+55RzlpV0ahnXXZJ27LtfQ0RERMTgMCCXWDSUmdL3U+XdA0wADrD9R0nvAU4C3gccDxxv+2xJB7Tp7oByzpmS3gwMAQ4F1rc9qtxvB2BNYFNAwMWStgb+j6rY3cf2f3Zx3nPArsDGVJ/9ncAdbcYzAtgSWAe4GDi/Bx/Ji7a3LGN9FHiH7ZfKUhGAw4CrbX+mtN0q6Q+2n6t3Imk/YD+AIcu3/HJnRERExKA1UAvkZSRNBkZSFZi/lzQM2Bw4T1LjvKXKv5sBO5XXZwGt1vzeBBwm6V+AX5ciu/mcHcrPXeX9MKpC+P+Ah23f3M15ywEX2n4eQNLFXTzjRbZfA+6WtEoX59WdW3s9FThT0kXARbVxfUzSl8v7pYG3AffUO7E9geqPDZYasWaSZCIiIqKjDNQC+QXboyQNBy6hWoN8GvBMY7a3t2yfJekW4MPAFZL+A3iw6TQBx9j+2XyN0kiq2eHuzjsY6GnB+VJTfwCvMv+ymKWbrqmP4cPA1sDHgG9IWq/08wnb9/VwDGyw2nBuzxffIiIiooMM6DXItmcBBwFfBl4AHpL0SZj7ZbmNyqk3A58or3dt1ZekdwIP2j6BaknDhsCzVLO+DVcAnymz1UhaTdI/teiu3XnXAjtLWkbScsBHe/nIDwPrSlqq/HHw/jbP8iZgddsTga9QfdlwWBnXgSpT45I27uX9IyIiIga9gTqDPJftuyRNoSp89wB+IunrwJLAOcAU4GDgl5K+BFwKzGrR1Tjg05JeAf4KHGn7KUk3lC/y/c72IZLeDdxUaszZwKeBOU1jurLVebbvlHQuMJmq2L2ul8/6Z0m/olo+8UfmLeFoNqQ873CqWeMf2H5G0lHAD4GppUieCXykN2OIiIiIGOxkD/4lppKGUi3LsKRdgd1sZweHHhgzZoyTpBcRERGDjaQ7bI9pdWzAzyD30GjgR2XW9BngM4t5PBERERHRTw3oNcg9Zfs62xvZ3tD21rYf6O4aSW8t+x9PlvRXSY/U3r95QcZR9jfuKpik+fy5+zS3OLaqpJ5s/RYRERERvdApM8i9ZvtJoLH/8XiaIqElLWH71cUxtnLvR4EeF9vd9NX2ORI1HREREZ2mI2aQ+0qZAf6+pInAsZLG1/YURtL0suUbkvYsKXZTJP2iRV9Hlf7e1NQ+ulxzE7UIbUl7SzpP0m+BKzV/CuAtZRu3xrmTSj8tk/Oa++rLzygiIiJioMsMcu+tBWxve06ZWX6dUqweBmxh+wlJKzYd/y4wnCp5r/lbkj8HDrR9jUq8dc1mwIZld42RtfZzgE8B35Q0AljV9h2SjqZFcl5zX717/IiIiIjBLTPIvXee7TndnPM+4HzbTwA0FaHfAFawvX9zcVy2ZVvB9jWlqXnm+fdtCtpfAZ8srz8FnFde7wAcWlIHJzEvOa+rvpC0n6TbJd0+5/lWO+JFREREDF6ZQe69elpdu2Q70T4x7zZgtKQVWxSoXV3XfO+5bD8i6UlJG1Lt57x/rb/XJedJek+7vkp/iZqOiIiIjpUCeeHMpARtSNoEeEdpvwq4UNIPbD/ZVAxfTpVod6mkHWw/2+ishHnMkrSl7eupgk966hyq1LzhtqeVtkZy3oFlD+iNbbcLF2kpUdMRERHRabLEYuFcAKxYljB8DrgfwPYM4NvANSXl7/v1i2yfB5wMXCxpmaY+9wF+XL6k90IvxnI+VZrgr2ptR1ElCk4tX+g7qhf9RURERHSkjkjSiwWXJL2IiIgYjLpK0ssMckRERERETQrkiIiIiIiaAVEgS5pTIp6nl4CLoYt7TN2RNFPSdU1tk9tFR0dERERE/zBQdrF4wXYj9vlM4ABqX3yTNKQHexN3qS/6aGE5Savb/rOkd/dx332iu+ceaFHTkLjpiIiIWDgDYga5yXXAuyRtK2mipLOAaZKWlvRzSdNKrPJ2AJKGSvpViX0+t8QyjynHZks6UtItwGaSDi+xzNMlTZCkct4kST+QdK2keySNlfRrSX+U9K0uxvorqn2JAXYDzm4cKFHR10m6s/xsXtq3Lfc7X9K9ks6sjaPd+MaW57tJ0nG1COoh5f1t5fj+tXvM/ez66hcTERERMRgMqAJZ0hLAB5lX1G0KHGZ7XeDzALY3oCpGT5e0NPCfwNO2N6Ta5mx0rctlgem231P2Hf6R7bG21weWoexxXLxse2vgp8Bvyv3WB/aW9NY2Qz4f+Hh5/VHgt7Vjfwc+YHsTqiL6hNqxjYGDgXWBdwJblPZ24/s5cIDtzYD6bPBngVm2xwJjgX0lNfZqrn9280mSXkRERHSygVIgL1P2Gr4d+D/gf0v7rbYfKq+3pEQz274XeBhYq7SfU9qnA1Nr/c6h2su4YbsywzyNKi56vdqxi8u/04AZth+z/RLwILB6m3E/BTwtaVfgHuD52rElgZPLvc6jKoYbbrX9F9uvAZOBke3GJ2kFYDnbN5Zzzqr1swOwZ/nsbgHeCqxZu8dDtGB7gu0xtscMGTq8zaNFREREDE4Dbg1yQ1ldUI9LVptr27UDvNhYf1tmm08CxpQ1w+OZFx0N8FL597Xa68b7rj7Hc4EfA3s3tX8R+BuwEdUfKi+2uBdURfwSXYyvq+cTcKDtK+ZrlLali6jpiIiIiE42UArknriWKpr5aklrAW8D7gOuBz4FTJS0LrBBm+sbxfATkoYBu1AtkVhYFwIjqGKfV621Dwf+Yvs1SXsBQ7rpp+X4bD8t6VlJ77V9M1WaXsMVwOckXW37lfK5PNKbwSdqOiIiIjrNYCqQTwJ+WpYfvArsbfslSSdRrUeeCtxFtcTidQtrbT8j6WSqJRQzgdv6YlC2nwWOhbmz3vXxXiDpk8BEupnR7WZ8n6VarvEcMIl5z3cK1fKMO8sX+h4HdlqoB4qIiIgY5AZ91LSkIcCStl+UtAZwFbCW7ZcX89D6jKRhtmeX14cCI2z/V1/0najpiIiIGIy6ipoeTDPI7QylWl6xJNWa3M8NpuK4+LCkr1L9Ph/m9eudIyIiIqKHBn2BXJY4tPzrYLCwfS7VlwEjIiIiYiEN+gJ5cZI0h/mDOHayPXMxDWeBJEkvIiIiOk0K5EXrddvT9YSkJWy/uigGFBERERFdGyhBIYOGpFGSbi7RzxdKektpnyTpaEnXAP8labSkayTdIekKSSNq53Ubey3ponLtDEn71dpnS/q2pCllHKu84R9CRERERD+WAnnRWkbS5PJzYWk7A/ifEn09Dfhm7fwVbG9DFTt9IrCL7dHAqcC3a+f1JPb6M+XaMcBBtfZlgZttb0S1d/S+zYNO1HRERER0siyxWLTmW2IhaThVEXxNaTqdKma6ofFFu7WpCt7fl72ThwCP1c57Xex16b8Re/0kVVG8czlvdaqI6SeBl4FLSvsdwAeaB217AjABYKkRaw7ufQAjIiIimqRA7l8aYSGiKnw3a3Nel7HXJUp6e2Az289LmsS8JL5XPG/z6znkfwMRERER80lx9AayPUvS05K2sn0d8O/ANS1OvQ9YWdJmtm8qezivZXtGD281HHi6FMfrAO9d0DEnajoiIiI6TQrkN95eVJHYQ4EHgX2aT7D9sqRdgBPKsowlgB8CPS2QLwcOKPHa9wE398nIIyIiIjrAoI+ajoWTqOmIiIgYjLqKms4uFhERERERNSmQIyIiIiJqsgZ5AbWIkT7H9nfKjhFftt2rdQmSRgGr2r6szfExwJ62D+qij8uA3W0/05t7dyVR0xEREdFpUiAvuAWKke7CKKpQj9cVyCV6+nagy6Lb9of6cDwRERERHSlLLBYhSTtIuknSnZLOkzSstI+VdGOJe7617FRxJDCupO6NkzRe0gRJVwJnSNpW0iXl+mGSfi5pWoms/kRpnylppfI6UdMRERERCyAF8oKrx0hPljSufrAUql8Htre9CdXs739LejNVYt5/lbjn7akCQg4HzrU9ynYjUW80sKPt3Zvu/Q1glu0NSmT11S3Gl6jpiIiIiAWQJRYLrrslFu8F1gVuKHHRbwZuooqRfsz2bQC2/wFQzml2se0XWrRvD+zaeGP76RbnJGo6IiIiYgGkQF50BPze9m7zNUobAj0tOp9r066u+ujLqOkk6UVERESnyRKLRedmYAtJ7wKQNFTSWsC9wKqSxpb25SQtATwLLNfDvq8EvtB4I+ktTcf7LGo6IiIiotOkQF5wzWuQv1M/aPtxYG/g7BL5fDOwju2XgXHAiZKmAL+nmt2dCKzbaj1zC98C3iJpeulju6bjlwNLlPseRaKmIyIiInosUdPRpURNR0RExGCUqOmIiIiIiB7q6AJZ0s6SXNbptjo+qSTY9bS/MZJO6MF5N3Zz/Gs9vWfTdQdLGrog10ZEREREpaOXWEj6FTACuMr2+BbHJ7EAsdF9MK7Ztoe1aBfV7+y1NtfNBMbYfqIX9xpie06740uNWNMj9vphT7vrFxI1HREREd3JEosWSqrdFsBnKXsKS1pG0jklne5cYJna+bMlHVvS6f4gadMyw/ygpI+Vc+ppd+MlnVo756B6X+XfEZKuLV/Mmy5pq/Jlv8YXAM+UNFLSPZJOAu4EVpf0kxLkMUPSEaWvg4BVgYmSJpa23Ura3nRJxzY9y5GSbgE2W3SfckRERMTA07EFMrATcLnt+4GnJG0CfA54vqTTfZsqya5hWWBSSad7lmoniQ8AO1PFRLeyDvCvwKbANyUt2XR8d+CKEjiyETDZ9qGUEBLbe5Tz1gbOsL2x7YeBw8pfPBsC20ja0PYJwKPAdra3k7QqcCzwPmAUMFbSTrVnmW77Pbav7+XnFhERETGodXKBvBtwTnl9Tnm/NfBLANtTgam181+m2j4NYBpwje1XyuuRbe5xqe2XypKHvwOrNB2/DdhH0nhgA9vPtunnYdv1rdo+JelO4C5gParEvmZjqQr6x22/CpxZng+qgJAL2twrUdMRERHR0ToySU/SW6lmVteXZGAIVTLdXbRPqKsn0L0GvARg+7US9NHKS7XXr0uts32tpK2BDwO/kHSc7TNa9DM3UU/SO4AvA2NtPy3pNOal5M33mG3GBPBiV+uOEzUdERERnawjC2RgF6olC/s3GiRdQ7XGdw+qdbzrUy1hWGQkvR14xPbJkpYFNgHOAF6RtGSZoW62PFXBPEvSKsAHgUnlWCON7wngFuB4SSsBT1PNkJ/Y2zEmajoiIiI6TacWyLsB32lquwDYmOoLclOBycCti3gc2wKHSHoFmA3sWdonAFPLMorD6hfYniLpLmAG8CBwQ+3wBOB3kh4r65C/SpXQJ+Ay279ZpE8TERERMQh09DZv0b0k6UVERMRglG3eIiIiIiJ6KAVyRERERERNvy2Qu4uB7sH1R0ravovjO0lqtT1aT/sfKWn3Lo5Z0lG1tpUkvSLpRwt6z4iIiIhY9Przl/R2A66nSrkb39uLbR/ezSk7AZcAd/d6ZJWRVEEfZ7U5/iDwEeAb5f0nqb5Y169IWqLsk9zStEdmMfLQS9/IIfWJxE1HRETEguqXM8itYqBLe6to5iGSTivvp0n6Yjn3NEm7lNffkXR3iZD+nqTNgY8Bx5W+1pC0r6TbJE2RdIGkobV+TpB0Y4mM3qUM5zvAVuX6L7Z4jBeAeyQ1Fn+PA35Ve5aPSrpF0l0lunqV0t5VRPVFqqKuZ0jar9b+WUn3l2tObsxSS1q5PMtt5WeL2j0mSLqSalu5iIiIiCj66wzy3BhoSU9J2sT2ncyLZv62pCHAUKoY5dVsrw8gaYV6R5JWpIqDXse2Ja1g+xlJFwOX2D6/nPeM7ZPL629RFeeNfYNHAFtSRUdfDJwPHAp82fZHuniOc4BdJf2VKijkUWDVcux64L1lTP8BfAX4Ujm2DrAd1Z7G90n6SdkT+TO2n5K0DHCbpAuApahmqTeh2gf5amBK6ed44Ae2r5f0NuAK4N3l2GhgS9svNA+6FN/7AQxZfuUuHi8iIiJi8OmvBfJuwA/L60YM9J1U0cynSloSuMj2ZEkPAu+UdCJwKXBlU1//AF4ETpF0KdWyilbWL4XxCsAwqmKy4SLbrwF3N2Z6e+hy4Cjgb8C5Tcf+BThX0gjgzcBDtWOX2n4JeElSI6L6L8BBknYu56wOrAn8M1Xs9VMAks4D1irnbA+sK80N1Vte0nLl9cWtimNIkl5ERER0tn63xKIWA32KpJnAIcA4SbJ9LbA18AhVNPOetp8GNqJKk/s8cEq9v7K+dlOqIJCdqIrWVk4DvmB7A+AI5o9vrkdGdxXhPB/bLwN3UM0MX9B0+ETgR+V++3dxvznAEpK2pSp4N7O9EVUs9tLdjOdN5fxR5Wc128+WY891cV1EREREx+qPM8jtYqC3lPR/NEUzS7oMeNn2BZL+RFXoUrt2GDDU9mWSbgYeKIcascwNywGPldnpPaiK8K40X9/O/6Oa4X2yNpMLMLx2j7160M9w4Gnbz5edPd5b2m8FfiDpLWVMnwCmlWNXAl8AjgOQNMr25B7ca65ETUdERESn6Y8FcrsY6N2Bm3l9NPNqwM8lNWbDv9p07XLAbyQ1ZlsbX6g7Bzi5fAluF6p1vLcAD1MVmN0Vv1OBVyVNAU6z/YNWJ9meQevdK8YD50l6pDzXO7q53+XAAapisO8r12D7EUlHl7E/SrUrx6xyzUHAj8s1SwDXAgd0c5+IiIiIjpao6UFA0jDbsyUtAVwInGr7wr7oO1HTERERMRgpUdOD3nhJk4HpVF/2u2gxjyciIiJiwOqPSyyil2x/eXGPISIiImKwSIG8ACT9M9U2dGOpdpyYCRxs+/5FeM+RVPs2ry9pFLCq7csWor+v2T66u/MGapIeJE0vIiIiFkyWWPSSqq0oLgQm2V7D9rrA16j2Kn6jjAI+tJB9fK0vBhIREREx2KRA7r3tgFds/7TRULZOu17ScbXI63EAkrYtEdDnS7pX0pmlyH5dBHZpmxuRXd7Prt9c0puBI6n2hp4saZykTUsU9l3l37XLuXtL+rWkyyX9UdJ3G/cFlinXn7lIP62IiIiIASZLLHpvfarwj2Yfp5rZ3QhYiSoK+tpybGNgPapt2G4AtpB0N00R2D25ue2XJR0OjLH9BQBJywNb235V0vbA0VT7IVPGtDHVUpD7JJ1o+1BJX7A9qtU9EjUdERERnSwzyH1nS+Bs23Ns/w24hmqNMsCttv9S4qonAyOZPwL748DzC3Hv4VR7Kk8HfkBVjDdcZXuW7Rep9kh+e3ed2Z5ge4ztMUOGDl+IYUVEREQMPCmQe28GMLpFe1eRz6+Lju4iAvtVyu+lLMV4cw/GdBQw0fb6wEfpJra6B/1FREREdKwUS713NXC0pH1tnwwgaSzwNNW64NOBFYGtgUOAdVp10kUE9kyqAvxXwI7Aki0ub465rsdW793D53hF0pK2X+nqpERNR0RERKfJDHIvuYoe3Bn4gKQ/SZpBFRt9FlX89BSqIvortv/aRVfLAZeUGOhrmBeBfTKwjaRbgfcAz7W4diKwbuNLesB3gWMk3QAM6eGjTACm5kt6EREREfNL1HR0KVHTERERMRglajoiIiIioodSIEdERERE1ORLegNUu7hrqjXQ99VO/b7tMyQNB04EtijtNwAH2p7V1X0SNR0RERGdJgXyAFSLuz7d9q6lbRRV3PWf2gSA/C8w3fae5fwjgFOAT74xo46IiIgYGFIgD0wt464ljWx1sqR3UW0dN67WfCTwgKQ1bP9pEY41IiIiYkBJgTwwtYu7BlhD0uTa+wOBtwCTbc9pNNqeU85bD5ivQE7UdERERHSyFMiDz+uWWEjaEWi1n59atdueQLVPMkuNWDP7AEZERERHSYE8MM0Adunl+RtLepPt1wAkvQnYCLinqwuTpBcRERGdJtu8DUxXA0tJ2rfRUOKu397qZNsPAHcBX681fx24sxyLiIiIiCIF8gDURdz1o5Q1yLWfg8plnwXWkvSApD8Ba5W2iIiIiKjJEosByvajwKdaHFqmzflPA59epIOKiIiIGAQygxwRERERUZMCuY9I+mdJ55QlD3dLukzSWgvQz8GShi7AdbPLvyMlTe/t9RERERFRyRKLPtBNst39vezuYOCXwPMt7jOkvpdxX+iuz0RNR0RERKfJDHLfaJlsB1wv6ThJ0yVNkzQOQNK2kiZJOl/SvZLOVOUgYNtCmoYAACAASURBVFVgoqSJ5dzZko6UdAuwmaT/Lv1Nl3RwV4OSNKTc/zZJUyXtX7v/RElnAdMWzUcSERERMTBlBrlvtEu2+zgwimq/4ZWA2yRdW45tTJVi9yhwA7CF7RMk/Tewne0nynnLAtNtHy5pNLAP8B6qkI9bJF1j+6424/osMMv2WElLATdIurIc2xRY3/ZDC/HcEREREYNOZpAXrS2Bs23Psf034BpgbDl2q+2/lOCOycDINn3MAS6o9Xeh7edszwZ+DWzVxf13APYskdK3AG8F1qzdv2VxLGk/SbdLun3O87N69KARERERg0VmkPtGu2Q7dXHNS7XXc2j/u3ixtka4q/5aEXCg7Svma5S2BZ5rd1GipiMiIqKTpUDuG1cDR0va1/bJMDfZ7mlgnKTTgRWBrYFDgHW66OtZYDngiRbHrgVOk/QdquJ3Z+Dfu+jrCuBzkq62/UrZVeOR3jxYoqYjIiKi06RA7gO2LWln4IeSDgVeBGZS7UgxDJgCGPiK7b9K6qpAngD8TtJjtrdrus+dkk4Dbi1Np3Sx/hjgFKqlG3eWnTYeB3bq7fNFREREdBJVqcURrY0ZM8a333774h5GRERERJ+SdIftMa2O5Ut6ERERERE1KZAjIiIiImpSIA9gkm5c3GOIiIiIGGzyJb1+rHyxTmWv5NexvXkf3GPQRk03JHI6IiIieiMzyAtA0rKSLpU0pUQ+j5M0U9JK5fgYSZPK6/GSfiHpakl/lLRvrZ9DajHQR5S2kZLukXQScCfwDUnfrV2zt6QTy+vZ5d8Rkq6VNLmMZ6vSvluJuJ4u6dhaH/PFVy/qzysiIiJiIEmBvGD+DXjU9ka21wcu7+b8DYEPUxWjh0taVdIOVKl2m1LFUY+WtHU5f23gDNsbAydRRVY3jAPObep/d+AK241Y68mSVgWOBd5X+h8rqbHFWyO++j22r28ebJL0IiIiopOlQF4w04DtJR0raSvb3VWRv7H9gu0ngIlURfEO5ecuqpnidZgXA/2w7ZsBbD8OPCjpvZLeSlU839DU/23APpLGAxvYfpYq0nqS7cdtvwqcSRVUAvPHV7+O7Qm2x9geM2To8O4/jYiIiIhBJGuQF4Dt+yWNBj4EHCPpSuBV5v3BsXTzJS3eCzjG9s/qBySN5PUx0OcCnwLuBS500+bVtq8ts88fBn4h6TjgH108wotdrTuOiIiI6GQpkBdAWb7wlO1flnXAe1Ml540Gfgd8oumSHSUdQ7W0YVvgUOAF4ChJZ9qeLWk14JU2t/w1cBjwMPA/LcbzduAR2ydLWhbYhGp5xfFlXfTTwG7Aib191kRNR0RERKdJgbxgNgCOk/QaVVH7OWAZ4H8lfQ24pen8W4FLgbcBR9l+FHhU0ruBm6rNKpgNfJpq+cN8bD8t6W5gXdu3Nh+nKroPkfRK6WdP249J+irVkg4Bl9n+zUI+d0RERMSgl6jpRaysC55t+3uLeywLIlHTERERMRglajoiIiIiooeyxGIRsz1+cY8hIiIiInouBXIPSTqMar/hOcBrwP62m9cav1FjubEvUvR6YjAk6dUlVS8iIiK6kwK5ByRtBnwE2MT2S2VniDcvwvst9ojpiIiIiE6VNcg9MwJ4wvZLALafKDtRkIjpiIiIiMElBXLPXAmsLul+SSdJ2qaH1w2oiOmGRE1HREREJ0uB3AO2Z1OFgOwHPA6cK2nvHlw6oCKma8+bqOmIiIjoWFmD3ENlve4kYJKkacBewGkkYjoiIiJiUEmB3AOS1gZes/3H0jSKKvYZBmnEdEOipiMiIqLTpEDumWHAiZJWoJoxfoBquQXAESRiOiIiImLQSNT0IjLQI6YbEjUdERERg1GipiMiIiIieihLLBaRRExHREREDEwDvkCWNAeYRrXedg7wBds3LsL7jQQusb1+N+eNB74JrGn7gdL2ReD7wFjbA2LdQqKmIyIiotMMhiUWL9geZXsj4KvAMc0nSBryxg8LqAr3XWvvdwHuXkxjaWsxfj4RERER/c5gKJDrlqfa2gxJ20qaKOksqkIVSRdJukPSDEmNXSgaEczfljRF0s2SVintq0i6sLRPkbR5uWSIpJNLP1dKWqbNeC4Cdix9vROYRRU00rjvT0pi3YxG5HRpnynpCEl3lsjodUr7ppJulHRX+Xft0j5U0q9KdPW5km6RNKYc20HSTaWv8yQNq93jcEnXA59c2A8+IiIiYrAYDAXyMpImS7oXOAU4qnZsU+Aw2+uW95+xPRoYAxxUEuqg2qf45jILfS2wb2k/AbimtG8CzCjtawI/tr0e8Ayv3/+44R/AnyWtT7UfcXNU9GHl25MbAttI2rB27AnbmwA/Ab5c2u4Fti5R1IcDR5f2/wSetr1hef7RAGUv5K8D25e+bgf+u3aPF21vafuc+qASNR0RERGdbMCvQaYssQCQtBlwRilIAW61/VDt3IMk7Vxer05V6D4JvAxcUtrvAD5QXr8P2BPmJunNkvQW4CHbk2vnj+xifOdQLbP4V+D9wD61Y58qM9lLACOAdYGp5diva/1/vLweDpwuaU2qVL4lS/uWwPFlnNMlNfp4b+nzhrLn8puBm2r3by7YKX1MACYALDVizewDGBERER1lMBTIc9m+qcyarlya5kY3S9oW2B7YzPbzkiYxLxr6lVqM8xy6/1xeqr2eA7RbYgHwW+A44Hbb/yiFKpLeQTUzPLYEgpzG/FHVjXvUx3MUMNH2zuXLgpMaj9fm3gJ+b3u3Nsebo61fJ0l6ERER0WkGwxKLucpa3SFUs8LNhlMtQ3i+nPfeHnR5FfC50vcQScv3dky2X6CKif5206HlqQrUWWXN8wd70N1w4JHyeu9a+/XAp8o41wU2KO03A1tIelc5NlTSWr19hoiIiIhOMhgK5MYa5MlUSwb2Ksshml0OLFGWHxxFVTx257+A7SRNo1rqsN6CDND2ObbvbGqbAtxFta75VOCGHnT1XeAYSTdQ/SHQcBKwcnm2/6FapjHL9uNUhfTZ5djNwDoL8gwRERERnSJR04NA2aZtSdsvSlqDauZ7LdsvL2zfiZqOiIiIwairqOlBtQa5gw0FJkpakmrd8ef6ojiOiIiI6ET9tkCWdBiwO9WX1F4D9rd9SxfnbwX8FHiFat3wW2xf9kaMtcVYGul+DefY/s4C9nWj7c27Osf2s1Rb10VERETEQuqXBXLZru0jwCa2Xyo7U7y5m8v2AL5n++eS9qYqGPusQJY0pM3a5lbmbj23sLorjheWpCVsv9rueKKmIyIiotP01y/pjaAKyngJwPYTth8FkPT+kiQ3TdKpkpaS9B9UuzgcLuls4EhgXPny3rhy7gqqPClpz9LXLyRtL2mkpOtK2tydjcQ8NaXxlZ0sjpN0W0mt2783D9VFQt7Kkn5f2n8m6eHyRwGSZtfGMknS+ZLulXSmyp5xkkZLukZVSuAVkkaU9jUkXV7ar6vd7zRJ35c0ETh2oX5TEREREYNMfy2QrwRWl3S/pJMkbQMgaWngNGCc7Q2oZsA/Z/sU4GLgkLLn7+HAubZH2T6XaoeILah2oXgQ2Krc571UOzv8HfhASZsbR5Wg11BP4/ss1e4QY4GxwL5lP+Nmc3fWaBTptWOtEvK+CVxd2i8E3tbmc9kYOJgq/OOdVFu4LQmcCOxSUgJPZd6WchOAA0v7l6l2u2hYiyph70vNN0mSXkRERHSyfrnEwvZsSaOpCtntgHMlHUq1LdpDtu8vp54OfB74YTddXgdsDTxMVZjuJ2k14Klyr+HAjySNolrzXN8ruJ7GtwOwoaRdyvvhVGl89bQ+6HqJRauEvC2BncuzXy7p6TbX3mr7LwBlW7uRVFHX6wO/LxPKQ4DHJA0DNgfOa4STAEvV+jqv3ZKRJOlFREREJ+uXBTLMjXaeBEwq+xDvBUzu8qL2rqUqpN8GHEZVjO5CVTgDfBH4G7AR1az6i7Vr62lzopqRvWIBxwGtE/LaJeG1u7Z+vYAZtjern1hCTZ7polDvNkUvIiIiohP1ywJZ0trAa7b/WJpGUc3+3guMlPQu2w8A/w5c06KLZ4HlGm9s/7nxRT/bD0q6nmrJwRfKKcOBv9h+TdJezB/CUXcF8DlJV9t+paTSPWJ7YYvNRhLesZJ2AN7Si2vvowoJ2axEbS9JtQfyDEkPSfqk7fPKeuUNS0BJjyVqOiIiIjpNf12DPAw4XdLdJQFuXWC87ReBfaiWDUyj2v7tpy2unwis27T+9xagsTTjOmA1qsIUqrW5e0m6mWp5RbuC9xTgbuBOSdOBn9H6j4zmNcjdbfF2BLCDpDupIqcfoyryu1X2O96FqrieQjXL3tj5Yg/gs6V9BrBjT/qMiIiI6GRJ0usHJC0FzLH9atni7id9tU3cwkqSXkRERAxGSpJev/c24FeS3gS8DOy7mMcTERER0bFSIPcDZa31xot7HBERERHRf9cgD1iS5pR1x9MlnSdpaB/0ObKsee5TJTBkl+7PjIiIiOgcmUHue3P3QJZ0JnAA8P3uLlI3kc+Ly2CLmq5L7HRERES0khnkRes64F2SPirpFlUR2X+QtAqApPGSJki6EjijXeR1Xbu4a0lvKqmDMyRdIumyxuywpMPL+dPL/Xq673JEREREx0mBvIhIWoJqy7ZpVNvJvdf2xsA5wFdqp44GdrS9O11HXje0i7v+OFWy3gbAfwD14JAf2R5re31gGeAj3Yw9UdMRERHRsbLEou8tU2KgoZpB/l9gbaq47BHAm5k/mvpi2y+U10vSPvK6oV3c9ZZU8dGvAX+VNLF2zXaSvgIMBVak2hP5t+0eIFHTERER0clSIPe9F5r3MJZ0IvB92xdL2hYYXztcDyXpKvJ6bne0iLuW1HJBraSlqYJQxpREwfHA0r15oIiIiIhOkgL5jTEceKS83qub87qLvG4Zd021jGMvSacDKwPbAmcxrxh+QtIwqtS983s68ERNR0RERKdJgfzGGE8Vj/0IcDPwjjbnnQRcIOmTVHHZrSKvT6Faa3xn+bLd48BOwAXA+4HpVJHat1CtVX5G0slUa6FnArf1zSNFREREDE6Jmh5EJA2zPVvSW4FbgS1s/3Vh+kzUdERERAxGiZruHJdIWoHqi4BHLWxxHBEREdGJUiAPIra3XdxjiIiIiBjoUiD3E5IOA3an2t7tNWB/27cs3lEN7iS9uqTqRUREREMK5H5A0mZU4R2b2H5J0kpUyyR6cu1CRVT314jriIiIiMUlBXL/MAJ4wvZLALafAJA0Gvg+MAx4Atjb9mOSJgE3AlsAV0vaB3hn2R5uKHAf8E7gbcCPqbZ9ex7Y1/a9kk4DngI2Bu4EvvRGPWhEREREf5eo6f7hSmB1SfdLOknSNpKWBE4EdrE9GjgV+HbtmhVsb2P7CGAKsE1p/yhwhe1XqNLwDizXf5lqG7mGtYDtbb+uOE7UdERERHSyzCD3A2VrttHAVsB2wLnAt4D1gd9X2x0zBHisdtm5Ta/HUe2dvCtwUgkF2Zxq/+XGeUvVrjnP9pw240nUdERERHSsFMj9RClWJwGTJE0DPg/MsL1Zm0vqISIXA8dIWhEYDVwNLAs80xx73eb6iIiIiChSIPcDktYGXrP9x9I0CrgH2EHSZrZvKksu1rI9o/n6MgN9K3A8cEkptv8h6SFJn7R9Xknd29D2lN6MLVHTERER0WmyBrl/GAacLuluSVOBdYHDgV2AYyVNASZTLZlo51zg08y/9GIP4LPl+hnAjoti8BERERGDSaKmo0uJmo6IiIjBqKuo6cwgR0RERETUpECOiIiIiKjpiC/ptYtxlnQwMMH284vgnpOowjre7rKORdJFVHsPD+vr+y0qnRI1DYmbjoiIiMqgn0FuinHeENge+HM5fDAwtJf9DenF6c9Qpd0haQWqxLx+RZVB/7+DiIiIiJ7qhMLodTHOth+VdBCwKjBR0kQASbtJmiZpuqRjGx1Imi3pSEm3AJtJ+rSkWyVNlvSzLormc6iCOwA+Dvy61ucwSVdJurPcc8fSPlLSPZJOljRD0pWSlinH9pV0m6Qpki4osdJIWkPSzeXYkZJm1+5zSGmfKumIpnucRBU1vfrCf8wRERERg0MnFMivi3EGsH0C8Ciwne3tJK0KHAu8j2of4rGSdip9LAtMt/0e4Emq1LotSgjHHKrt1Fq5Cti6FNC7Mv8WbC8CO9vehCo97/9pXuTdmsCPba9HNQv9idL+a9tjbW9EtU/yZ0v78cDxtseWZwJA0g6lr03LM42WtHU5vDZwhu2NbT9cH3SipiMiIqKTDfoC2fZsqnS5/YDHgXMl7d3i1LHAJNuP234VOBNoFJNzgAvK6/eX/m6TNLm8f2eb288BrqcqqJexPbN2TMDRZd/jPwCrAauUYw/Znlxe3wGMLK/Xl3RdSdrbA1ivtG8GnFden1W7xw7l5y6qmeJ1qApmgIdt39xq0LYn2B5je8yQocPbPFpERETE4NQRX9JrEeO8F3Ba02mivRdLH43zTrf91R7e/hzgQmB8U/sewMrAaNuvSJoJLF2OvVQ7bw6wTHl9GrCT7SmlyN+2m3sLOMb2z+ZrlEbSw6jpJOlFREREpxn0M8iS1pa0Zq1pFNBYUvAssFx5fQuwjaSVypKI3YBrWnR5FbCLpH8q/a8o6e1dDOE64Bjg7Kb24cDfS3G8HdBVHw3LAY+V2On6so6bmbcMY9da+xXAZyQNK2NdrTHuiIiIiGitE2aQhwEnll0kXgUeoFpuATAB+J2kx8o65K8CE6lmXi+z/ZvmzmzfLenrwJVl94dXgM8zr+huPt/A91ocOhP4raTbqWKk7+3Bs3yDqpB/GJjGvOL+YOCXkr4EXArMKve+UtK7gZvK8ubZVHHUc4iIiIiIlhI1PQiU3SxesG1JuwK72d6xL/pO1HREREQMRl1FTXfCDHInGA38qOyC8QzwmcU8noiIiIgBa9CvQX4jSbKkX9TeLyHpcUmXLGB/MyWt1N15tq+zvZHtDW1vbfuBFn1dVpaZREREREQXMoPct56j2optGdsvAB8AHlnMYwLA9ocW5LpETUdERESnyQxy3/sd0Ki0dqO2e4WkZSWdWpLt7qql5w2R9L2SqDdV0oG1/g6spe2tU87fVNKNpY8bJa1d2veW9GtJl0v6o6Tv1u49dzZa0p7lPlPqM94RERERkQJ5UTgH2FXS0sCGVLtONBwGXF0S77YDjpO0LNWuGu8ANra9IdUOFw1PlLS9nwBfLm33Alvb3hg4HDi6dv4oqmCSDYBxkuaLkZa0XhnH+0oi3381P0CS9CIiIqKTZYlFH7M9tQRx7AZc1nR4B+BjkhqF7tLA24DtgZ+WBD9sP1W75tfl3zuAj5fXw4HTy/7OBpasnX+V7VkAku6m2l/5z7Xj7wPOt/1Ei3s1nmEC1RZ4LDVizWxzEhERER0lBfKicTHV3sfbAm+ttQv4hO376ieX3SfaFaKNVL05zPt9HQVMtL1zKcYntTi/+Zr6GFL0RkRERLSRAnnROBWYZXuapG1r7VdQrSk+sOxZvLHtu4ArgQMkTbL9qqQVW83s1gxn3pf/9u7l2K4CLpT0A9tPdnevRE1HREREp8ka5EXA9l9sH9/i0FFUyyGmSppe3gOcAvxfaZ8C7N7NLb4LHCPpBmBIL8c2A/g2cE251/d7c31ERETEYJckvehSkvQiIiJiMOoqSS8zyBERERERNSmQIyIiIiJqUiAPIpJOk7TL4h5HRERExECWXSyiS50UNQ2Jm46IiIjMIPd7JZ760hILPV3SOEmHl7jq6ZImlH2Um68bLekaSXdIukLSiNJ+kKS7S9T0OW/8E0VERET0b5lB7v/+DXjU9ocBJA0Hfm/7yPL+F8BHgN82LpC0JHAisKPtxyWNo9ra7TPAocA7bL8kaYVWN5S0H1X8NUOWX3mRPVhEREREf5QZ5P5vGrC9pGMlbVVipLeTdIukaVTR0es1XbM2sD7we0mTga8D/1KOTQXOlPRp4NVWN7Q9wfYY22OGDB2+KJ4pIiIiot/KDHI/Z/t+SaOBD1GFg1wJfB4YY/vPksYDSzddJmCG7c1adPlhYGvgY8A3JK1nu2WhHBEREdGJUiD3c5JWBZ6y/UtJs5kXLf2EpGHALsD5TZfdB6wsaTPbN5UlF2sB9wCr254o6XqqxL5hwDPt7p+o6YiIiOg0KZD7vw2A4yS9BrwCfA7YiWrpxUzgtuYLbL9ctns7oaxZXgL4IXA/8MvSJuAHttsWxxERERGdKFHT0aVETUdERMRglKjpiIiIiIgeSoEcEREREVHTL9YgS5pDtaa24Rzb31nE9xwJbG77rPJ+DLCn7YP6qP/jqb5At7rt1xbg+suA3dutEZZ0MDDB9vMLOL5tgZdt39jVeZ2WpNeQRL2IiIjO1S8KZOAF26Pe4HuOpNrF4SwA27cDfbLYVtKbgJ2BP1NtqTapt33Y/lA3pxwM/BJYoAIZ2BaYDXRZIEdERER0mn67xELScEn3SVq7vD9b0r7l9SElanmqpCNq1+xZ2qaUhDkknVZ2dGicM7u8/A6wlaTJkr4oaVtJl0h6k6SZ9ZQ5SQ9IWkXSypIuKPe+TdIWbYa/HTAd+AmwW62fbcr9Jku6S9JykkZIura0TZe0VTl3pqSV2kRNHwSsCkyUNLGc/xNJt0ua0fSZzJR0xP9n777j7Krq9Y9/HkLvoogRS5AiQsBAAkoVELl2QDp4FVERCyhX8IcNo1wUBEEFUQNCAENvRlSKSO8QUggiSLEgVwSkRGrC8/tjrUN2hjkzk2SSTOY879drXnPO2nuvvc4Jf3xnsfZ6JE2QNEXS2nX2fD/gwHrfLebsXykiIiJi8BkoM8hL1cS3lu/ZPlvSF4CxdbnCq2yfKGk7YE1gY8pWZeMlbQk8Bnwd2Mz2o5JW6uWehwAH2f4gvLzkANsvSfoVZQb4FEnvAB60/U9JZ1C2RrtO0puAS4G3ddP3HsCZwK+A70pazPaLwEHA521fX/cwfo4S6Xyp7cMlDQGW7tLXK6KmbT8p6X+ArW0/Ws/7uu3Hax9XSFrf9uR67FHbG0r6XP3Mn5L0M2Ca7aO7Dj5R0xEREdHJBsoM8rO2RzR+zgawfTllbfJPgE/Vc7erP3cAE4C1KQXzNsB5rYLR9uNzMZ6zgd3q693re4BtgeNrMT8eWF7Scs0LJS1OSb27yPZTwM11vADXA8fUGeAVa4LdrcAnaiLeeraf7jKW7qKmu7OrpAmU72VdYJ3GsQvq79spS0t6lKjpiIiI6GQDpUDuVl3L+zbgWaA1IyzKDHOrmF7D9i9qe3ebOk+nfk5JAhbvw61vBNaQtDIllKNVYC4CbNK496rdFLTvBVYApkh6ENicusyiPnj4KWAp4CZJa9u+hrJO+SHgdEkfa3Zm+x5gJKVQ/p6kQ7v5nlajzE6/2/b6wG+YNX76+fp7BgPn/xpEREREDEgDvVg6kBKP/DXgZEmbUJY1HCZpnO1pklalJMxdAVwo6Vjbj0laqc4iP0gpMM8BtgcWq30/DSxHN2xb0oXAMcAfbT9WD10GfAE4CkDSCNsTu1y+B/Ap22fWc5YBHpC0NDDU9hRK8bwJsLakZ4GH6vKRZYANgdNanal91HRr/I8CywP/AZ6UtArwPnp/MPDpel2PEjUdERERnWagFMhd1yBfApxMmW3d2PbTkq4BvmH7W5LeBtxYJoSZBnzU9lRJhwNXq2wbdwelmDwR+JWkWyhF9H/qPSYD0yVNAsbW85vOpix/2LvRdgDwE0mTKd/dNZSH3QCoRfB/AZ9ptdn+j6TrgA8Bm0vamjKTexfwO8oSjoMlvVg/yywzyHQfNQ0wBvidpIdtby3pDmAqcD9lKUdvfg2cJ2l7YH/b1/bhmoiIiIhBL1HT0aNETUdERMRgpERNR0RERET0TQrkiIiIiIiGgbIGueNI2pGyO8bbbN89h32MBS62fd5sXrc3cJntf/R2bqdGTfckMdQRERGDW2aQF5w9gOsoD+nNb3tTkvgiIiIioosUyAtATdHbDPgktUCuUddXSTpP0t2SxtV9m5F0aI22vlPSmFZ7o793123pWu/fI+kCSUNUorbvrDHTB6rEbo8CxtWY6aXm2wePiIiIWAikQF4wdgAuqSEgj0vasLZvAHyJkoL3FkoRDXC87Y1sD6eEjHywS39/AN5Wg00APgGcAowAVrU93PZ6wCl1OcZtwF417OTZroOTtK+k2yTdNuOZdsF9EREREYNTCuQFYw/grPr6rPoe4Bbbf7f9EjCRmbHQW0u6WdIUSqT2us3OXPbqOx34qKQVgU0oeyzfD7xF0nGS3gs81ZfBJWo6IiIiOlke0pvPJL2aUuQOl2RgCCUi+7fMjISGGgstaUngBGCU7b9JGs2sMdItp1DCP54DzrU9Hfi3pLdTwks+D+wK7DM7402SXkRERHSazCDPfzsDp9l+s+1htt8IPABs3ub8VjH8aF27vHN3J9UdKf4BfIOSDIik1wCL2D4f+CYlxhp6iNmOiIiI6HSZQZ7/9gCO6NJ2PiVC+r6uJ9t+QtKJwBTgQUr8dTvjgJVt31XfrwqcIqn1h9BX6++xwM8kPQts0t065IiIiIhOlajpQUTS8cAdtn/RX30majoiIiIGo56ipjODPEhIuh34D/DlBT2WiIiIiIVZ1iDPAUmvrnsIT5T0f5IearxfvA/XbyXp4tm854clHdLuuO2RwKK2n293TkRERET0LjPIc8D2Y5Q9hqm7SkyzffQ8vud4YHwv52za3/dN1PQrJWo6IiJicMsMcj+RNFLS1ZJul3SppKG1fQ1Jv5c0SdIESavXS5Ztk5r3oKRv13OnSFq7tu9d1xgjaRVJF9Y+J0natLZPq7+XlXRFo4/ta/swSX+UdKKkqZIuS5JeRERExKxSIPcPAccBO9elDicDh9dj44Cf2H47sCnwcG1vl5oH8KjtDYGfAgd1c78fA1fXPjcEfH7bSAAAIABJREFUpnY5/hywY+1ja+AHjXjqNet41gWeAHZ6xYdJkl5ERER0sCyx6B9LAMOBy2sdOgR4WNJylKjnCwFsPwdQz7nF9t/r+1Zq3nW1vwvq79uBj3Rzv22Aj9U+ZwBdq1gB35W0JfASZbu3VeqxB2xPbPQ/rGvntscAYwCWGLpmtjmJiIiIjpICuX8ImGp7k1kapeV7uOYVqXndHOva3ld7ASsDI22/KOlBZgaOdL1vllhERERENKRA7h/PAytL2sT2jZIWA9ayPVXS3yXtYPsiSUtQZpfn1hWUYJEfShoCLGP7qcbxFYBHanG8NfDmOb1RoqYjIiKi02QNcv94iRIBfaSkScBEynpjgP8GDpA0GbgBeF0/3O+LwNaSplCWSazb5fg4YJSk2yizyXf3wz0jIiIiOkKS9KJHSdKLiIiIwainJL3MIEdERERENKRAjoiIiIhoWOgKZEk7SnIrQGMBj2UHSeu0OTa6jnONRtuBta3b6fyIiIiIWPAWxl0s9qDsF7w7MHrBDoUdgIuBu9ocn0IZ5//W9zv3cO4CI2lI3U/5FRI1/UqJmo6IiBjcFqoZZEnLUhLnPkkpPFvtQyQdXWOVJ0vav7ZvJOmGGsd8i6TlJC0p6ZR67h11G7RZopzr+4slbVVfT5N0eO3nphr1vCnwYeAoSRMbEdJNFwGtmOe3UAI9/tW4x09rYt1USd9utLeLm964fp476u+31valJZ1TP/vZkm5uzVJL2k7SjbWvc+t32LrHoZKuA3aZy3+aiIiIiEFjoSqQKTO2l9i+B3hc0oa1fV9gNWAD2+sD4yQtDpwNfLFGMm8LPAt8HsD2epTZ6FMlLUnPlgFuqv1cA3za9g3AeOBg2yNs39fNdU8Bf5M0vN7r7C7Hv16fnlwfeJek9RvHuoubvhvY0vYGwKHAd2v754B/189+GDASQNJrgG8A29a+bgP+p3GP52xvbvus5qASNR0RERGdbGErkPcAWsXcWfU9lOL3Z7anA9h+HHgr8LDtW2vbU/X45sDpte1u4C/AWr3c9wXKUgpoE8/cg7Mos907ABd2ObarpAnAHZS9jJvrmZtx0637rQCcK+lO4Fhm7n+8eb0Ptu8EJtf2d9Y+r69x1h9n1tCQrgU7tY8xtkfZHjVk6RX6/kkjIiIiBoGFZg2ypFcD2wDDJZmSSGdJX6FEPXfd0Lm7tlZ7d6Yz6x8MzVnlFz1zw+jZjX/+NXAUcJvtp6Rye0mrUWaGN7L9b0lju9yzu7jpw4Arbe8oaRhwVS+fScDltvdoc/w/s/E5IiIiIjrCQlMgUx5wO832Z1oNkq6mzJ5eBuwn6Srb0yWtRFmO8HpJG9m+VdJylCUW11DS5f4gaS3gTcCfgOWBz0laBFgV2LgPY3oaWK6nE2w/K+n/Afd0ObQ8pUB9UtIqwPuYWfC2swLwUH29d6P9OmBX4Mq6q8Z6tf0m4CeS1rD9Z0lLA2+oS1T6JFHTERER0WkWpiUWe/DKJQrnA3sCJwF/BSbXqOc9bb8A7AYcV9sup8zQngAMqTHNZwN7234euB54gLLzxNHAhD6M6Szg4PrQXHcP6QFg+yzbE7q0TaIsrZgKnFzv35vvA9+TdD1lBr3lBGDlGmf9/yhLLJ60/S9KIX1mPXYTsMC3x4uIiIgYyBI1PQhIGgIsZvu5WqhfAaxV/0iYK4majoiIiMGop6jphWmJRbS3NGV5xWKUdcef7Y/iOCIiIqITpUAeBGw/DSSdLyIiIqIfdHyBLGkGZd3xosAfgY/bfqbNuXsDo2x/QdJ+wDO2T5vL+68AHEcJQIGyFnl/20/W40cB7wd+S1mDfDGwOHAA8FXKeusn5mYMPUmSXntJ1IuIiBicFqaH9OaVZ2vQx3DKfsf79eUi2z+b2+K4+gVwv+3Vba9OeVDwpMbxzwAb2j4YeDdwt+0NbF9r+/1zWxxL6vg/kiIiIiKaUiDP6lpgDUkrSbqoRjff1CXhDgBJoyUdVF+vIen3NYp6QmtHC0kHS7q19vPtbvpYg5J6d1ij+TvAKEmrSxpPSfG7uW4V933g/TXaeqkaF/2a2tfH6n0mSTq9tq0s6fw6hlslbdYY+xhJlwH9UeRHREREDBqZPazqTOr7gEuAbwN32N5B0jaUInJED5ePA46wfWGNrV5E0nbAmpT9lAWMl7Sl7Wsa160DTLQ9o9Vge0ZNvVvX9oclTbM9oo7xn9QlHvV9a+zrAl8HNrP9aN0HGuBHwLG2r5P0JuBS4G312Ehgc9vPdvNd7EuJ72bI8iv34duLiIiIGDxSIMNStSCFMoP8C+BmYCcA23+Q9Oq6VvgVagDJqrYvrOc/V9u3A7aj7HUMsCylYG4WyD2l/c3O/nvbAOfZfrSO4fHavi2wTquQBpav4wUY311xXK8fA4wBWGLomtkHMCIiIjpKCuS6BrnZoEZF2dCuUOwp5vl7tn/ew72nAhtIWsT2S/XeiwBvpzww2FftCupFgE26FsL14yVmOiIiIqIbKZC714qjPkzSVsCjtp/qrm6u7X+XtIPtiyQtQUm5u7ReP872NEmrAi/afqRx7Z8l3QF8g7L2mPp6gu0/z8Z4rwAulHSs7cckrVRnkS8DvgAcBSBphO2JPXXUVaKmIyIiotPkIb3ujaY8KDcZOAL4eC/n/zdwQD3/BuB1ti8DzgBurLHW5wHLdXPtJ4G1JP1Z0n3AWrWtz2xPBQ4Hrq6x2sfUQwe0Poeku+jjDh0RERERnSxR09GjRE1HRETEYNRT1HRmkCMiIiIiGlIgR0REREQ05CG9bkh6NeXBN4DXATOAf9X3G9t+YQ76HAtcbPu8Xs55F/AUsBRwE/BV2w+1Of8k4Bjbd83uePoqUdPtJWo6IiJicEqB3A3bj1GDQSSNBqbZPrp1XNKitqfPo9sfbPu8utXcl4ArJQ3vWpRLGmL7U/1xw9rXjN7PjIiIiBj8ssSijySNlXSMpCuBI5tR0/X4nZKG1deviH3u0tdhtb+237+LY4H/oyT8IWmapO9IuhnYRNJVkkZJ+qyk7zf631vScfX1RyXdUuOpfy5pSHd99cNXFBERETEopECePWsB29r+crsTGrHP29h+O/DFLse/D7wW+EQrHKQXE4C16+tlgDttv8P2dY1zzgM+0ni/G3C2pLfV15vVMJQZlP2de+oLSftKuk3SbTOeebIPQ4yIiIgYPFIgz55z+7AUoV3sM8A3gRVtf8Z931+vmU4yAzi/6wm2/wXcL+mddf30W4HrgXcDI4Fba5z2u4G39NRX7W+M7VG2Rw1ZutuE7YiIiIhBK2uQZ08znnk6s/6BsWT93S72GeBWYGQj6a4vNmDmA4PP9VCgnw3sCtwNXGjbdR3zqba/2s35PfX1siTpRURERKfJDPKcexDYEEDShsBqtf0KYNc6k4uklRrXXEJJ5vuNpO5S9V6m4gBgaL2uNxcAOwB7UIrl1lh2lvTa1lgkvbkPfUVERER0rBTIc+58YKW6dOGzwD3QY+wz9fi5wInAeElLddPvUfW6e4CNgK37sq2c7X8DdwFvtn1LbbsL+AZwWY3BvpxScEdEREREG4majh4lajoiIiIGo0RNR0RERET0UQrkeUzSjpIsae36foSk9zeObyVp0wU3woiIiIhoyi4W894ewHXA7sBoSkLfKOC39fhWwDTghr52OI+T/GaRqOmeJW46IiJi8EmBPA9JWhbYDNia8lDed4HvAEtJ2hw4E9gPmCHpo8D+wF+Bk4GVgX9RAkX+Kmks8Dhl27cJdZeMpyjF9uuAr9g+r973YMqWb0tQtnz7lqTDgEdt/6ieczjwT9s/ng9fRURERMRCIwXyvLUDcInteyQ9DgwHDgVG2f4CQN3JYprto+v7XwOn2T5V0j7Aj2s/MDPJb0YtmIcCm1OS9sYD50naDlgT2JiyJ/N4SVsCv6BsBfejGnG9ez3nFSTtC+wLMGT5lfvz+4iIiIgY8LIGed7aAzirvj6rvu/NJsAZ9fXplAK4pWuS30W2X6rbua1S27arP3cwM6Z6TdsPAo9J2qB13PZj3Q0gSXoRERHRyTKDPI/UJRDbAMMlGRhCSdj71mx21dyH7z9djj3fvGXj9/ds/7ybvk4C9qYsyTh5NscRERER0RFSIM87O1OWSnym1SDpauBNQDNF72lg+cb7GyjLH04H9qI84Dc7LgUOkzTO9jRJqwIv2n4EuJCyBnoxYM++dJao6YiIiOg0WWIx7+xBKUibzqfM3q4jaaKk3YBfAzvW91sABwCfqMl3/w18cXZuavsyyhKNGyVNAc6jFuQ1ke9K4JwuSzUiIiIiokqSXgepD+dNAHaxfW9frkmSXkRERAxGSdILJK0D/Bm4oq/FcUREREQnyhrkDlF3unjLgh5HRERExEDXpxnkrnHJbc65SlK309Rtzh8lqdeQCkk9JsxJ+lpf79nlui9JWrrx/reSVpyTvrrpe4P6ff3XHF6/n6SP9XB8ruKpJa0o6XNzen1ERETEYNanNciSzqGEUlxhe3Sbc64CDrI9XxesSppme9lu2kX5fC+1ue5BSmDHo/NgTN+n7Gd8n+2950H/o2mEi8zB9cOAi20P7+3cJYau6aEf/+Gc3KZjJG46IiJi4TNXa5AbccmfpGw/1mpfStJZkiZLOhtYqnFsmqQjJd0u6feSNq4zzPdL+nA9ZytJF9fXoyWd3DjngGZf9fdQSdfU3R7ulLSFpCMosc0TJY2TNEzSHyWdQHkY7Y2SfirpNklTJX279nUA8HrgSklX1rYHJb2mjvtzjfuPlvTl+vpgSbfWz/ztNt+XKFu87Q1sJ2nJ2r6MpN9ImlTHv1ttP0LSXbXPoxv3PKg11sbxs2pxux9wYGvnC0kfknSzpDvq971KL9/rEcDq9fqjevtvICIiIqKT9GUN8ixxyZI2tD0B+CzwjO31Ja1PKUhblgGusv3/JF0I/C/wHmAd4FRKLHJXawNbU7Yk+5Okn9p+sXF8T+BS24dLGgIsbftaSV+wPQJenhl9K/AJ25+rbV+3/Xi95gpJ69v+saT/AbbuZgb5LOCHwAn1/a7Ae9Umwtn2NV2u3wx4wPZ9dVb9/ZSI5/cC/7D9gTquFSStBOwIrG3bbZZ4HAKsZvt5SSvafkLSz5g1nvpVwDtrH58CvgJ8ud33Wvsc3vreulKipiMiIqKD9WUNcru45C2BXwLYngxMblzzAnBJfT0FuLoWu1OAYW3u8xvbz9eC9RFmRie33ErZH3g0sJ7tp9v08xfbNzXe7yppAiV6eV1Kkd6W7TuA10p6vaS3A/+2/VfaRDh300W772sKsG2dod7C9pPAU8BzwEmSPgI8001/k4Fxkj4KTG8z7DcAl6rse3xw/ZwtvX2v3X0HiZqOiIiIjtVjgayZcckn1TW7BwO71WUEMGsMctOLnrm4+SVqJHJdD9xu1roZmzyj63l1pnZL4CHgdLV/iO3lOGZJqwEHAe+2vT7wG2DJNtc1nUdZJrEbM4vdVoTziPqzhu1fNC+qs9Q7AYfW7+s44H2SlrN9DzCSUih/T9KhtqdTZqTPp87UdzOWDwA/qdfeLqm77+844Hjb6wGf6fIZe/xeIyIiImJWvRVL7eKSNweuoUQhXylpOLD+PBtlue+bgYdsnyhpGWBD4DTgRUmLdVmO0bI8pWB+sq7LfR9wVT32NGXZQXcP6Z0FnAi8BnhXbespwrllW2CS7Zd3r5B0KrCDpCuAx23/sq6r3ltlfffStn8r6SbKPsXNz7wI8EbbV0q6jrLMZFleGU+9AuUPB4CPd/N5ump99l4lajoiIiI6TW8F8h6UB7qazqcUav8DnKISiTwRuKX/hzeLrYCDJb0ITANaM8hjgMl1GcXXmxfYniTpDmAqcD9wfePwGOB3kh62vXWX66ZKWo5SkD9c2y6T9DZKhDN1DB+lLFtoaRcv/dl63lGSXgJerG3LAb+qD/IJOLDLtUOAX0paoR4/tq5B/jVwnqTtgf2B0cC5kh4CbgJWa/Mdtj7fY5Kul3Qn8DvbB/d0fkREREQnSdR09ChR0xERETEYKVHTERERERF9kwI5IiIiIqIhOxoMQJIMHGO7FVByELBsuxTDeWnKQ08y7JDfzO/bLrSSqhcREbHwywzywPQ88BFJr5mTi+t2cxERERExB1IgD0zTKbtsdN3VAkljJe3ceN+K4t5K0pWSzgCm9BBtPVLS1Sox4JdKGjqfPlNERETEQiFLLAaun1C2r/v+bFyzMSVC+gFJO/HKaOvFKKEi29v+Vy2aDwf2aXaSqOmIiIjoZCmQByjbT0k6DTgAeLaPl91i+4H6egpwtKQjgYttX1sDXYYDl9e9nIcAD3dz7zGUGWyWGLpm9gGMiIiIjpICeWD7ITABOKXRNp26NKZGfi/eOPZyzLbteySNBN5Piba+jBJiMtX2JvN64BERERELqxTIA5jtxyWdA3wSOLk2PwiMBM4BtgcW6+5aSa+nS7Q1JRVxZUmb2L6xLrlYy/bUdmNI1HRERER0mjykN/D9AGjuZnEi8C5JtwDvoDFr3MV6wC2SJlIiuP/X9gvAzsCRkiZRIsI3nWcjj4iIiFgIJWo6epSo6YiIiBiMEjUdEREREdFHKZAjIiIiIhrykN4AJ2k0MM320V3a9wOesX3avLx/oqbnTqKnIyIiFj4pkBdCkha1/bN+7Gt6f/QVERERMRikQB6AJH0d+BjwN+BfwO2SrgJuADYDxktaDpgG/AY41fbG9dphwHjb69d9kI8BlgUeBfa2/XDXvig7ZUREREQEKZAHnFrU7g5sQPn3mQDcXg+vaPtd9bzRALb/KGlxSW+xfT+wG3BOH2KlX+6rmzEkajoiIiI6VgrkgWcL4ELbzwBIGt84dnaba84BdqUEgexWf95Kz7HS7fpK1HRERER0tBTIA1O7orRdKMjZwLmSLgBs+15J69FzrHS7vmaRJL2IiIjoNNnmbeC5BthR0lJ1nfGHervA9n3ADOCbzJwZ/hM1VhpA0mKS1p1HY46IiIgYNDKDPMDYniDpbEoM9F+Aa/t46dnAUcBqtZ8XJO0M/FjSCpR/6x8CU/t/1BERERGDR6Kmo0eJmo6IiIjBKFHTERERERF9NOgLZElflzRV0mRJEyW9o5fzx9alCXNzz60kWdInG20b1LaD5qbviIiIiJi3BvUa5PqA2geBDW0/L+k1wOLz6fZTKNut/aK+3x2YNJ/u3We9JeklanruJW46IiJi4TLYZ5CHAo/afh7A9qO2/wEg6VBJt0q6U9IY1c2CmySNlHS1pNslXSppaG0/QNJddVb6rDb3/iuwpKRVat/vBX7X6PvT9f6TJJ0vaenaPlbSjyXdIOn+1my2pGUlXSFpgqQpkrZv9PVNSXdLulzSma1ZakmrS7qkjv9aSWs37nGMpCuBI+f2S46IiIgYTAZ7gXwZ8EZJ90g6QVIzOe542xvZHg4sRZlpflkjiW5n2yOBkylJdACHABvYXh/Yr4f7nwfsAmxKScR7vnHsgnr/twN/BD7ZODYU2LyO6Yja9hywo+0Nga2BH6gYBexESd77CNBcbD4G2L+O/yDghMaxtYBtbX+566Al7SvpNkm3zXjmyR4+XkRERMTgM6iXWNieVqObt6AUlWdLOsT2WGBrSV8BlgZWomx/9uvG5T0l0U0Gxkm6CLiohyGcQ9l+bW3gTEqh3DJc0v8CKwLLApc2jl1k+yXgLkmr1DYB35W0JfASsCqwCqWQ/pXtZwEk/br+Xrbe79zG5PgSjXuca3tGd4NOkl5ERER0skFdIAPUIvAq4CpJU4CP12URJwCjbP9N0mhgyS6XivZJdB8AtgQ+DHxT0rrdreO1/X+SXgTeA3yRWQvkscAOtidJ2hvYqnGsOdPcqm73AlYGRtp+UdKDdcyvWBpSLQI8YXtEm+N9StKLiIiI6DSDukCW9FbgJdv31qYRlPCNVjH8aJ1p3ZmyHKLp5SQ62zfWJRdrUZZDvNH2lZKuA/akzAA/0WYYhwKvtT2jyzLn5YCHa797AQ/18nFWAB6pxfHWwJtr+3XAzyV9j/Lv+QHgRNtPSXpA0i62z63roNe3PVsPCiZqOiIiIjrNoC6QKYXrcZJWBKYDfwb2tf2EpBMpO008CNza9cIekujuAX5Z2wQca7tdcYztG9oc+iZwM6Vgn0IpmHsyDvi1pNsoKXt31/5vlTSeskPGX4DbgNbC4b2An0r6BrAYcBYDcCeNiIiIiIEkSXqDgKRl63rrpYFrKH8ETOiPvpOkFxEREYNRT0l6g30GuVOMkbQOZenIqf1VHEdERER0ohTIg4DtPRf0GCIiIiIGi8G+D3K/kbRjjYpeewGPY4c6Wzyn1w+TlII6IiIioo3MIPfdHpQdI3YHRi/AcewAXAzcNYfXD6PsvHFGX05O1PS8kfjpiIiIgSszyH1Qt4LbjJJ2t3ttGyLp6Br7PFnS/rV9oxoTPUnSLZKWk7SkpFPquXfUbdqQtLek4xv3uVjSVvX1NEmH135uqpHVm1L2Xj5K0sQaJT1bkdWUZL4t6vUHzp9vMCIiImLhkQK5b3YALrF9D/C4pA2BfYHVmBk5PU7S4pTkvC/WCOltgWeBzwPYXo8yE32qpK7BJF0tA9xU+7kG+HTdMm48cLDtEbbvY/Yjqw8Brq3XH9vdjRM1HREREZ0sBXLf7EHZQ5j6ew9K8fuzVoKe7ccp8dQP2761tj1Vj28OnF7b7qbsV7xWL/d8gbKUAuB2ytKI7gyXdG1NCdwLWLdx7CLbL9m+ixJL3Se2x9geZXvUkKVX6OtlEREREYNC1iD3QtKrgW0ohaiBIYApRWvXTaTVTVurvTvTmfWPlOas8oueuUn1DNr/W41l9iKrIyIiIqIHKZB7tzNwmu3PtBokXQ1MAPaTdJXt6ZJWoqTbvV7SRjXhbjnKEotrKLO7f5C0FvAmSpT18sDnJC0CrAps3IfxPM2sqXuzG1nd9foeJWo6IiIiOk2WWPRuD+DCLm3nA68H/gpMljQJ2NP2C8BulHjrScDllFnhE4AhdRnE2cDetp8HrgceoERNH00puntzFnBwfdhvdWZGVl9OjZ/uxWRgen2oLw/pRURERHSRqOnoUaKmIyIiYjDqKWo6M8gREREREQ0pkCMiIiIiGjrmIT1JVwHfs31po+1LlO3WLgHWsX1Em8tn5x5DgeeAacA+tv80l31+B7jG9u+7tG8FHGT7g3PY79dsf7e385KkN28lUS8iImLg6aQZ5DOpKXgNuwNn2h4/t8Vxw141tONU4KiuByUNmZ3ObB/atTjuJ1+bB31GRERELPQ6qUA+D/igpCUAJA2j7ERxXTPyuUY6X1h3eZhU452R9NEaHT1R0s/7UOheA6xRr50m6TuSbgY26a6v+jNW0p01kvrAeu3YVky0pPdKulvSdcBHWjeStIykk2vk9B2Stq/te0u6QNIlku6V9P3afgSwVL3/uP75eiMiIiIGh44pkG0/BtwCvLc27Q6c7Vdu4/Fj4Oo6C7whMFXS2yjbt21mewQluGOvXm75Icr2bVBio++0/Q7gsTZ9jQBWtT28RlKf0uysRlOfWPvdAnhd4/DXgT/Y3gjYGjhK0jL12Ih6v/WA3SS90fYhwLM1bvoVnyNR0xEREdHJOqZArprLLHav77vaBvgpgO0Ztp8E3g2MBG6VNLG+f0ube4yr52wGHFTbZlD2TqaHvu4H3iLpOEnvBZ7q0u/awAO2761F/S8bx7YDDqn9XUXZe/lN9dgVtp+0/RxwF/DmNuN+WaKmIyIiopN1zEN61UXAMZI2BJay3ZdgDigxzafa/mofzt3LdteNg5+zPaO3viS9Hfgv4PPArsA+XU5pt2m1gJ26PhAo6R3MGjfdU2R1RERERNBhxZLtaXWniZPpfvYY4Args8AP6zrjZWrbryQda/uRGiu9nO2/zMEwuu0L+A/wgu3zJd0HjO1y3d3AapJWt30fJeGv5VJgf0n727akDWzf0cs4XpS0mO0XezopUdMRERHRaTptiQWUwvjtlMjm7nwR2LrGQt8OrGv7LuAbwGWSJlNinYfOyc176GtV4Kq6TGIs8NUu1z0H7Av8pj6k1yzODwMWo8Re31nf92ZMPT8P6UVEREQ0JGo6epSo6YiIiBiMEjUdEREREdFHKZAjIiIiIho66iG9BU3SisCetk/oxz5HA9NsH91ffTYlanr+Sex0RETEwJAZ5PlrReBzC3oQTbMbfR0REREx2KVAnr+OAFavEc9HSTq4xkNPlvTt1kmSLpJ0u6SpkvZttL9X0oQagX1Fo991JF0l6X5JBzTO7zYeu2v09Xz43BERERELjRTI89chwH01YvpyYE1gY0oc9EhJW9bz9rE9EhgFHCDp1ZJWpkRN71RjsHdp9Ls2JWBkY+BbkhbrJR775ehr29d1HWSipiMiIqKTZQ3ygrNd/WkFeixLKZivoRTFO9b2N9b2lYFrbD8AYPvxRl+/sf088LykR4BVmDXSGmAp4JF6fjP6+hVsj6Hsk8wSQ9fMPoARERHRUVIgLzgCvmf757M0SlsB2wKb2H6mJv8tWc9vV6x2FyfdUzx2M/q6R0nSi4iIiE6TJRbz19OUWGko8dD7SFoWQNKqkl4LrAD8uxbHawPvrOffCLxL0mr1/JV6udcVwM61TyStJOnN/ftxIiIiIgafzCDPR7Yfk3R9jYP+HXAGcGNdAjEN+ChwCbBfjaH+E3BTvfZf9YG9CyQtQlku8Z4e7nWXpFak9SLAi8DnmTWiOiIiIiK6SNR09ChR0xERETEYJWo6IiIiIqKP5nuBLOkNkn4l6V5J90n6kaTF58N995b0+sb7kyStM4/uNUySJR3WaHuNpBclHT8v7hkRERER/WO+rkFWWWx7AfBT29vX4IoxwOHAwf3Q/5AedmfYG7gT+AeA7U/N7f16cT/wQeCb9f0uwNR5fM/ZJmlR29PbHU/U9PyXyOk6KDaTAAAgAElEQVSIiIgFa37PIG9D2WLsFIBazB5I2c1h6TrL+ytJl0j6k6RvtS7sayqcpENrOt2dksao2JkSujGuXr9UTZ4bVfvYQ9KUes2RjXtOk3R4Ta67SdIqtX2Xeu4kSde0+azPAn9s3YMS2nFOo+8PSbpZ0h2Sft/oe7Skk9sk47VL2PukpHvqNSe2ZqklrSzp/Pp93Cpps8Y9xki6DDhtDv4dIyIiIgat+V0grwvc3myw/RTwV2CN2rQxJfFtBLCLpFGzmQp3vO2NbA+nhGN80PZ5wG3AXrZH2H62df+67OJISvE+AthI0g6Nvm+qyXXXAJ+u7YcC/1XbP9zD5z0L2F3SG+qY/9E4dh3wTtsb1PO+0jj2imS82t5dwt7rKbPU76TsarF2o58fAcfa3gjYCTipcWwksL3tPbsOOkl6ERER0cnm9zZv7cIumu2X234MQNIFwObAdPqeCre1pK8ASwMrUZY1/LqHMW0EXGX7X/We44AtgYuAF4CL63m3M3NbteuBsZLOoSwZaecS4DDgn8DZXY69AThb0lBgceCBxrHukvH+TvcJe68Drm4l60k6F1irnrMtsE79zgCWl9Tah3l88w+FpiTpRURERCeb3wXyVMpM5sskLU8p9u6jFMFdCzLTx1Q4SUsCJwCjbP9N0mhKCl1P1MOxFz1zH7xWQh2295P0DuADwERJI1pF/SwDt1+QdDvwZcrs+Ycah48DjrE9XiU9b3Tj2CuS8dRzwl47i9TzZymEa8H8nx6ui4iIiOhY87tAvgI4QtLHbJ9W1xH/ABhbiz6A96ikxD0L7ADsAzwD/ErSsbYfqceXs9019KJVDD+qklC3M3BebWum2DXdDPxI0muAfwN7UIrXtiStbvtm4GZJH6IU+K8okKsfUGZ4H2vM5EJJzHuovv54T/drnN9dwt4twLGSXkX5jDsBU+qxy4AvAEfVcY+wPbEP93pZoqYjIiKi08zXNch1NnZHytrie4F7gOeArzVOuw44HZgInG/7Ntt3Aa1UuMnA5cDQbvp/AjiRUiBeBNzaODwW+FnrIb3GNQ8DXwWuBCYBE2z/qpePclTroT7K2uRJPXzmqbZP7ebQaOBcSdcCj/ZyPyjLNRatn/8wZibsPQR8l1Lo/x64C2gtHD4AGCVpsqS7gP36cJ+IiIiIjjagkvQk7U1ZHvGFBT2WhYmkZW1Pk7QocCFwsu0L+6PvJOlFRETEYKQk6Q16oyVNpOzz/ABl9jwiIiIi5sD8XoPcI9tjKUshYjbYPmhBjyEiIiJisMgMckREREREw4CaQY75T4maXugkijoiImLeygzyPCBpmKS7JZ1UI6nHSdpW0vWS7pW0cf25oUZN3yDprfXavSVdoBK3fa+k7zf6/WlNuJsq6duN9vfX+10n6ceSLq7ty9TY6lvrfbZv3ONcSb+mbAUXEREREVVmkOedNYBdgH0p283tSUkF/DBlW7uPAVvani5pW8pWba0QlRHABpTAkD9JOs7234Cv23687h99haT1KVvl/bz29YCkMxtj+DrwB9v7SFoRuEXS7+uxTYD1Wwl8TZL2reNmyPIr99f3EREREbFQSIE87zxgewqApKnAFbYtaQowjBL8caqkNSlpgYs1rr3C9pP12ruANwN/A3atxeuilH2g16H8X4D7bbeiqs+kFrfAdsCHJbUe4lsSeFN9fXl3xTEkajoiIiI6WwrkeacZF/1S4/1LlO/9MOBK2ztKGgZc1ebaVtT0asBBwEa2/y1pLL1HTQvYyfafZmksMdmJmo6IiIjoRgrkBacZNb13H85fnlLUPilpFeB9lKL6buAtkobZfhDYrXHNpcD+kvavs9cb2L5jdgaZqOmIiIjoNHlIb8H5PvA9SdcDQ3o72fYk4A5gKnAycH1tfxb4HHCJpOuAfzIzavowytKNyTUW+7D+/hARERERg82AipqOOdOImhbwE+Be28f2R9+Jmo6IiIjBKFHTg9+na9T0VMrSjZ8v4PFERERELLSyBnkQqLPF/TJjHBEREdHpUiDPA5JmAFMaTWfZPmIO+7rB9qb9M7LZlyS9hVcS9yIiIuZMCuR541nbI/qjo3ldHPcWNR0RERHRabIGeT6S9KCkb0uaIGmKpLVr+8qSLq/tP5f0F0mvqcem1d9bSbpK0nk1VnpcfSgPSSMlXS3pdkmXShpa21evkdW3S7q2cb+xko6RdCVw5AL5MiIiIiIGqBTI88ZSkiY2fpp7Ez9qe0Pgp5TgD4BvUSKhNwQuZGbaXVcbAF+iJOi9BdhM0mLAccDOtkdStoA7vJ4/Bti/th8EnNDoay1gW9tf7noTSftKuk3SbTOeebLr4YiIiIhBLUss5o2ellhcUH/fDnykvt4c2BHA9iWS/t3m2lts/x2g7loxDHgCGA5cXieUhwAPS1oW2BQ4t7YDLNHo61zbM7q7SaKmIyIiopOlQJ7/WjHSM5j5/fcUF93dtc3rBUy1vUnzREnLA0/0UKgnajoiIiKiGymQB4brgF2BIyVtB7xqNq79E7CypE1s31iXXKxle6qkByTtYvvcul55/ZrI12eJmo6IiIhOkzXI80bXNci9bfH2bWA7SROA9wEPA0/35Ua2XwB2phTXk4CJlKUVAHsBn6ztU4Ht5+CzRERERHSURE0PAJKWAGbYni5pE+Cn/bVN3NxK1HREREQMRj1FTWeJxcDwJuAcSYsALwCfXsDjiYiIiOhYKZAHANv3UrZwi4iIiIgFLAVyP2pETC8K/BH4uO1n5rLP3wJ72n6iS/toYJrto+egz2HAprbP6O3cRE0v/BI5HRERMXvykF7/etb2CNvDKUsl9mselDRkdju0/f6uxXE/GAbs2c99RkRERAwKKZDnnWuBNWpE9JWSzgCmSBoi6ShJt0qaLOkzAJKGSrqm7npxp6QtavuDjdjpr0v6k6TfA29t3aiXSOkfS7pB0v2Sdq6XHAFsUe914Hz8TiIiIiIGvCyxmAckLUrZru2S2rQxMNz2A5L2BZ60vVHdveJ6SZdRUvUutX14nWleukufI4HdKWuVFwUmUNL4oKTe7Wf7XknvoERKb1OPDaUk9a0NjAfOAw4BDrL9wTbj3xfYF2DI8ivP3ZcRERERsZBJgdy/lqoR0FBmkH9B2ZP4FtsP1PbtgPUbs7krAGsCtwIn16CPi2xPZFZbABe21jRLGl9/9xYpfZHtl4C7JK3Slw+RqOmIiIjoZCmQ+9ezXfcvrkVrM9ZZwP62L+16saQtgQ8Ap0s6yvZpXU7prlhdhJ4jpZvx1H2NtH5ZkvQiIiKi02QN8vx3KfDZOlOMpLUkLSPpzcAjtk+kzDxv2OW6a4AdJS0laTngQwC2nwIekLRL7U+S3t7LGJ4Gluu/jxQRERExeKRAnv9OAu4CJki6E/g5ZSZ/K2CipDuAnYAfNS+yPQE4mxIlfT5lCUfL7EZKTwamS5qUh/QiIiIiZpWo6ehRoqYjIiJiMOopajozyBERERERDSmQIyIiIiIasotFh5O0qO3p7Y4narozJZ46IiI6WWaQBxhJ/1OT9O6U9CVJX5F0QD12rKQ/1NfvlvTL+nqapMPrQ3c3tfY7lrSypPNrat+tkjar7aMljakBJV23kouIiIjoaCmQB5CalvcJ4B3AO4FPU3ar2KKeMgpYtm4Rtzkzd7JYBrjJ9tsp28F9urb/CDjW9kaUnTFOatxuJLC97T27Gce+km6TdNuMZ57sz48YERERMeBlicXAsjklLe8/AJIuoMRUj6x7Hz9PiZgeRSmaD6jXvQBcXF/fDrynvt4WWKeRsLd87QdgvO1nuxtEkvQiIiKik6VAHli6S7oz8CBlZvkGyh7GWwOrA3+s57zomfv1zWDmv+siwCZdC+Fu0v0iIiIiokqBPLBcA4yVdASlWN4R+G9gJeAgYB9gCnAMcLt738T6MuALwFEAkkbYnjg7A0rUdERERHSarEEeQGpa3ljgFuBm4CTbd1DWGg8FbrT9T+A5Zk3Sa+cAYJSkyZLuAvabJwOPiIiIGESSpBc9SpJeREREDEZJ0ouIiIiI6KMUyBERERERDSmQ5xNJMyRNlDS1Bnr8j6RF6rFRkn7cT/f5Wn/0ExEREdGpsgZ5PpE0zfay9fVrgTOA621/q4dreoyB7u0+fTxflP8OXuru+BJD1/TQj/9wdoYQg1giqCMiYrDIGuQBxvYjwL7AF1RsJelieGUMdA9x0ctKOkXSlLpLxU51e7il6kz1uHreLNHVtW2YpD9KOoESPPLGBfE9RERERAxE2Qd5AbF9f11i8dpuDo8ENrf9rKQzKHHR10l6E3Ap8Dbgm8CTttcDkPQq2+dL+oLtEbWtGV0t4GZJVwP/Bt4KfML257reXNK+lAKeIcuv3L8fPCIiImKAS4G8YHWXnAezxkC3i4veFti91Wj7393001109RbAeOAvtm/q7uaJmo6IiIhOlgJ5AZH0Fkos9COUGeGmZgx0u7hoUWKoe7xND8cSNR0RERHRjRTIC4CklYGfAcfbdmN2uDvt4qJb7a11xa+qs8gvSlrM9ou0j67us0RNR0RERKfJQ3rzT+vhuanA7ykF7rf7cF27uOj/BV5VH76bBGxd28cAkyWN6yG6OiIiIiLayDZv0aNETUdERMRglG3eIiIiIiL6KAVyRERERETDoH1IT9IbgJ8A61D+ELgYONj2C/X4mcC6wCnA74CzKLtC7AycbnvTubz/3rXvbW1fUdt2BC4AdrF93tz0P79MeehJhh3ymwU9jBhAkqYXERGD3aCcQa5boF0AXGR7TWAtYFng8Hr8dcCmtte3fSywA/Ar2xvYvm9ui+OGKcAejfe7A5P6qe9+I2nQ/qEUERERMbsGZYEMbAM8Z/sUANszgAOBfSQtTdlB4rV1V4lvUbZK+5SkKwEkTWt1JOkrNc55Ut0uDUmrS7pE0u2SrpW0dptxXAtsLGkxScsCawATG30fWuOj76zx0qrtV0k6UtItku6RtEVtH1bvN6H+bFrbF5F0gqSpki6W9FtJO9djIyVdXcd6qaShjXt8tybrfbF/vvaIiIiIhd9gnTlcF7i92WD7KUl/pRSpHwYubkQyC5hm++jmNZLeR5ldfoftZyStVA+NAfazfa+kdwAnUIryrkzZ0u2/gBUoCXarNY4fb/s79V6nAx8Efl2PLWp7Y0nvB75FSc57BHiP7eckrQmcCYwCPgIMA9ajRFf/EThZ0mLAccD2tv8laTfKLPo+9R4r2n5X10EnajoiIiI62WAtkNulzPUlfa5pW+AU288A2H68zgRvCpzbCPhYooc+zqLsZbwC8GXga41jW0v6CrA0sBIwlZkF8gX19+2U4hdgMeB4SSMoKXxr1fbNgXNtvwT8X2smHHgrMBy4vI51CPBw4/5ndzfgRE1HREREJxusBfJUYKdmg6TlgTcC91FmWfuiu4J6EeCJ1uxzb2zfImk48Kzte1pFtaQlKTPPo2z/TdJoYMnGpc/X3zOY+e90IPBP4O11HM81xtlu/FNtb9LmeOKmIyIiIroYrAXyFcARkj5m+zRJQ4AfAGPrUom+9nMZcKikM1pLLOos8gOSdrF9bl2esb7tnh6++yozi9mWVjH8aJ2V3hnobWeLFYC/235J0scpM8IA1wEfl3QqsDKwFXAG8CdgZUmb2L6xLrlYy/bUvn38RE1HRERE5xmUD+m5xAPuCOwi6V7gHkqB+rUeL3xlP5dQ1g3fJmkicFA9tBfwyRrxPBXYvpd+fmf7yi5tTwAnUna6uAi4tQ9DOoFSCN9EWV7RmgE+H/g7cCfwc0qs9JN1S7udgSPrWCdSlodERERERBuJmh4kJC1re5qkVwO3AJvZ/r+57TdR0xERETEY9RQ1PViXWHSiiyWtCCwOHNYfxXFEREREJ1rollhIWkXSGZLur3v73lgT6hbUePaWdHybdkt6d6Ntx9q2c3+Pw/ZWtkfYXsf22P7uPyIiIqJTLFQzyPWBuIuAU23vWdveTNnXeF7ed1Hb0+fg0laS3hX1/YBN0mv3+RI1Hb1J9HRERAw2C9sM8jbAC7Z/1mqw/RfbxwFIGiLpqJpON1nSZ2r7VjU57jxJd0sa10it61PSnKQPSbpZ0h2Sfi9plT6MN0l6EREREQuZhWoGmZKQN6GH45+k7N6wkaQlgOslXVaPbVCv/wdwPbCZpJvpY9KcpFcB77RtSZ8CvkIJ/ujJQpmkFxEREdHJFrYCeRaSfkJJkXvB9kbAdsD6jTW+KwBrAi8At9j+e71uIqWgfIK+J829ATi7zsAuDjzQx2EudEl6StR0REREdLCFrUCeJSHP9uclvQZo7UMmYH/blzYvkrQVM5PpYGY63ewkzR0HHGN7fO1vdF8GvDAm6SVqOiIiIjrZwlYg/4H/396dh9lV1Vkf/64upkAgKKIdBon4BmyJMSEBjAOTNt0IAjYg0NhCOyDqi9DthJ1XG+UVRXBgENqoCCoIBlGRbkwiEEBkSshEJAEloAwKiAlEkCGu/uPsIieVupVblVRu1a31eZ566tx9pn1/IWHXrn3PgtMkfcD2+aVt09r+acAHJF1r+zlJOwEP9nC93iTNjahd65he9jtJehERERGDxKAaIJf1v4cAXylLEx6lmgX9RDnkm1TLEe4oH3h7FDikh+s9W5ZjnC1pBFU9vko1U93VKcBUSQ8Ct7DqWuI19fvqbtqWSupM0ruP5pP0fijpcOA6Vk3SezNVkt7d1JL0evH+IiIiIoIk6bUNJUkvIiIiomlKkt6QkCS9iIiIiHUgA+Q2YXvvVvchIiIioh0MtqCQbklaIWluCdyYKmnTNZ/1wrkTJZ3dx/ue1OheJYjjt53hH6Xtx5KW9+VeEREREbF+tMsM8tO2xwFIuhg4Hvhy505JHbZXdHei7VmsfExcb50EfA94qsH+pcAbgF+U5Q8j+3ifflMG8CrPUF5NoqZjoEm0dURE9Le2mEHu4kbg/5R46eskXQIskLSJpG9LWlDioveBF2Koryrbm0m6oMQ/z5F0cGnvkHRmOXe+pBMkfRjYBriuFszR1aXAkWX7n1gZ/oGk4ZKuKZHRC2r3GiXpLknfKNHR0yUNK/veV/o2T9IPO2evJb1S0i1l32frs9SSPqaV0duf6XKP86iSCbdfN6WPiIiIGPzaaoAsaQNgf6pHpwHsDky2/WrgQwC2XwMcBVxUwjrqJgPXllS+fYAzJG1GlSr3CmC87bHAxbbPpoqt3sf2Pg26dA2wp6QOqoFyPbnuL8Dbbe9a7vWl2nKM0cDXbO9CNQvdGY5yhe3dbL+WKk76PaX9LOCs0u+HavXYr1xrd2AcMEHSnmX3zsB3bI+3fX+XOh4naZakWSueWtbgrUVERES0p3YZIA8r8dGzgN8C3yrtt9nujIR+I/BdANuLgPtZGdXcaT/g5HKtmVRhHi8H3gL8l+3ny/mPN9mvFVQhHkcAw2zfV9snqtCT+cDPgW2Bl5V9S2zPLdv1qOkxkm6UtAA4GtiltE8CppbtS7q8n/2AOVQzxa+iGjAD3G/7lu46bXuK7Ym2J3ZsOqLJtxoRERHRHtpuDXKnMhlbj1JuFMe8ymnAobYXd7mWgL4+MPpS4EesHk19NFXq3YSS+ncfK9P1usZiDyvbFwKH2J4n6ViqxLyeCPi87a+v0iiNokHMdERERMRQ1y4D5GbcQDUovbZEUL+cKop5Uu2YacAJkk4oqX3jbc8BpgPHS5pp+3lJLy6zyE8CmwOP9XDfG4HPA9/v0j4CeKQMjvcBdmjiPWwOPFwio49mZfT1LVTLMC5j5ZrnzvdzqqSLS4jItsBzTdznBYmajoiIiKGmXZZYNOM8oKMsT7gMONZ250xt5+zwqcCGwHxJd5bXUEVY/7a0zwP+ubRPAa7u4UN6uHKm7a6D6IuBiZJmUQ12FzXxHj5FFSM9o8vxJwH/Luk2qidlLCv3nk615OLm8r4vpxpkR0REREQDQz5qWtKhwEG2j2l1X/qqPM3i6TLrfSRwlO2D18W1EzUdERER7ShR0w1IOgj4HPDuVvdlLU0Azi1rpZcy+N9PRERERMsM6QGy7SuBK1vdj7Vl+0bgta3uR0REREQ7GEprkAclSZNLYMj8Eqe9Rw/HXijpsPXZv4iIiIh2M6RnkAc6SZOAA4FdbT8j6SXARuuzD4majui9xGFHRAxumUEe2EYCj3U+bcP2Y7YfkvTpEh99p6QptQS+F0iaIOl6SbMlTZM0srR/WNKvyoz0pev5/UREREQMeBkgD2zTge0l3S3pPEl7lfZzS+T0GKoQkQPrJ5XnJJ8DHGZ7AnAB1YcRAU5mZWT28d3dNFHTERERMZRlicUAVsI9JgBvAvYBLpN0MvCkpI8DmwIvBhYCP62dujMwBphRJpc7gIfLvvnAxZJ+DPy4wX2nUD3jmY1Hjh7azwGMiIiIIScD5AHO9gpgJjCzhH28HxgLTLT9O0mnsDKiupOAhbYnsboDgD2Bg4BPSdrF9vP91f+IiIiIwSYD5AFM0s7AX23fU5rGUcVjjwUekzQcOIwqIa9uMbC1pEm2by5LLnYC7gK2t32dpF9QJQIOp3p2crcSNR0RERFDTQbIA9tw4BxJWwLPA78GjqMa0C4A7gNu73qS7WfL497OljSC6s/5q8DdwPdKm4Cv2G44OI6IiIgYioZ81HT0LFHTERER0Y56iprOUywiIiIiImoyQI6IiIiIqMka5DYjaQXV+uQNgCXAv9heKmlUef1h2+eUY88FZtm+sNH1kqQX0RpJ44uIaJ3MILefp22PKyEijwMfqu17BDhR0nqNq46IiIgYTDJAbm83A9vWXj8KXAMc05ruRERERAx8GSC3KUkdwJuBK7vs+gLwkbK/0bmJmo6IiIghKwPk9jNM0lzgj1Qx1DPqO20vAW6jCgnplu0ptifantix6Yh+7WxERETEQJMBcvt52vY4YAdgI1Zdg9zpNOAT5M8/IiIiYjV5ikWbsr1M0oeBn0g6v8u+RZJ+BRxINZvcUKKmIyIiYqjJDGIbsz0HmAcc2c3uzwHbrd8eRURERAx8mUFuM7aHd3n9ttrLMbX2eeQHpIiIiIjVZIAUEREREVGTAXIfSJosaaGk+ZLmStqjRf34ZSvuGxEREdHOssSilyRNovpw2662n5H0EqqnRfTHvQTI9l+722/79evgHh22VzTan6jpiNZL7HRExPqVGeTeGwk8ZvsZANuP2X5I0n1lsIykiZJmlu1TJH1X0rWS7pH0vs4LSfqYpNvLTPRnStsoSXdJOg+4A/iUpC/WzjlW0jlle3n5PlLSDWU2+05JbyrtR0laUNpOr11juaTPSroVmNSv1YqIiIgYZDJA7r3pwPaS7pZ0nqS9mjhnLHAA1WD005K2kbQfMBrYHRgHTJC0Zzl+Z+A7tscD5wH/VLvWEcBlXa7/z8C08vzj1wJzJW0DnA7sW66/m6RDyvGbAXfa3sP2L3r17iMiIiLaXAbIvWR7OTABOA54FLhM0rFrOO0ntp+2/RhwHdWgeL/yNYdqpvhVVANmgPtt31Lu9yhwr6TXSdqKavB8U5fr3w78q6RTgNfYfhLYDZhp+1HbzwMXA50D8BXADxt1NlHTERERMZRlDXIflDW7M4GZkhYAxwDPs/IHjk26ntLNawGft/31+g5Jo4A/dzn+MuAdwCLgR7ZXuZ7tG8rs8wHAdyWdATzRw1v4S0/rjm1PAaYAbDxydNe+R0RERLS1DJB7SdLOwF9t31OaxgH3A8OoZpavBg7tctrBkj5PtbRhb+Bk4GngVEkX214uaVvguQa3vQKYXO7ziW76tAPwoO1vSNoM2JVqecVZZV30n4CjgHN6+36TpBcRERFDTQbIvTccOEfSllSzxr+mWm7xd8C3JP0HcGuXc24D/ht4OXCq7YeAhyT9HXBz9bAKlgPvpFr+sArbfyrR0K+23V009N7AxyQ9V67zLtsPS/ok1ZIOAf9j+ydr99YjIiIi2p+6/LY+1rGyLni57TNb3Ze+mDhxomfNmtXqbkRERESsU5Jm257Y3b58SC8iIiIioiZLLPqZ7VNa3YeIiIiIaN6gm0GWtKIWiPHTsha4L9fZRtLlPezfUtIH+97TNd7/FEkfbdD+lKSX1tqW91c/IiIiImJVg3EG+ekSiIGki4APAZ/r7UXKB+UO6+GQLYEPUgV1NGVN0dC98BjwEbp5YkWT/digPPt4rSVqOmJgSNx0RMT6M+hmkLu4GdgWQNIrJf1M0mxJN0p6Va39lhLp/NlaPPMoSXeW7V0k3VZmpudLGg18AXhlaTujHNdMNPT23R1Xjp0sabGkn1MFfjRyAXCEpBfXG+t9Lq8/Wj4EiKSZkk6TdD1woqTDyyz7PEk3lGM6JJ1R69v7+176iIiIiPY0GGeQgWqwB7wZ+FZpmgIcb/seSXtQzfzuC5wFnGX7+5KOb3C548sxF0vaCOigelbxmNpsdT0aWsCVJZzjt1SD3X+1/cEejvszcCQwnqrudwCzG/RnOdUg+UTgP3tRli1t71X6uwD4B9sP1pahvAdYZns3SRsDN0mabntJ/SKSjqN6dB0dW2zdi9tHREREDH6DcYA8TNJcYBTVAHOGpOHA64Gp5ZnCABuX75OAQ8r2JUB3j1u7GZgsaTvgijLI7npMPRoaquchj6YaIL8QDd3DcZtTpeA9BSDpyjW8z7OBuZK+tIbj6i6rbd8EXCjpB1RBI519Gyupc2nJiNK3VQbISdKLiIiIoWwwDpCftj1O0gjgKqo1yBcCSztne3vL9iWSbqWKap4m6b3AvV0OazYautFxJ7F65HRPfVoq6RKqddCd6nHWsHqk9Qv9sH18mUk/gGqgPa707QTb05rtR0RERMRQMxgHyADYXibpw8BPgPOBJZIOtz21fFhurO15wC1U0c+XUS1xWI2kHYF7bZ9dtscC86hmfTtNo7lo6EbH3UA1o/sFqrq/Dfh6N+fXfbocgNwAAA2cSURBVBm4nZV/Tn8AXippK6plGAcCP2vwnl5p+1bgVklvA7YvffuApGttPydpJ6qI6j93dw1I1HREREQMPYN2gAxge46keVQD36OB8yX9P2BD4FKqQe5JwPckfYQq7nlZN5c6AnhniWr+PfBZ249Luql8KO5q2x9rJhra9vTujrN9h6TLgLnA/cCNTby/xyT9CPi38vo5SZ+lirJeAizq4fQzyocNBVxTajGfamnKHeWHiEdZufwkIiIiIhgCUdOSNqValmFJRwJH2T641f0aLBI1HREREe1IPURND+oZ5CZNAM4tM6ZLgXe3uD8RERERMYC1/QDZ9o3Aa1vdj4iIiIgYHNp+gDyYSFoBLKBaQ/08cBHw1e6S+SRtA5xtu6c0wLWWJL2IwSWJexERay8D5IGlHqP9UqrnNo+gS1hIiZJeU1R2U9ZlLHVEREREOxjsUdNty/YjVGl2/1eVYyVNlfRTYHqXqOxbJe3SeW6JnZ4gaTNJF5Ro6TmSDi77V7lWK95fRERExECVGeQBzPa9kv4GeGlpmkT1fOfHS0BJp0uBdwD/KWkksI3t2ZJOA661/e4SN32bpJ93vVbX+yZqOiIiIoayzCAPfPXM6xndDWiBHwCHl+13AFPL9n7AySWaeyZV8t7L13AtbE+xPdH2xI5NR6xt/yMiIiIGlcwgD2Al1W8F8Ehp6jbxzvaDkv4oaSxV6Mn7Oy8BHGp7cZfr7tHoWhERERFDXQbIA5SkrYH/As4tISdrOuVS4OPACNsLSts04ARJJ5RrjLc9pzf9SNR0REREDDVZYjGwDJM0V9JC4OdUH6D7TJPnXk4Vuf2DWtupVI+Mm18+0HfquuxsRERERDtq+6jpWDuJmo6IiIh21FPUdGaQIyIiIiJqMkCOiIiIiKjJh/QGCEnbAV8DXk31g8tVwMdsP9vKfiVqOiKGisR0R0SnzCAPAKoeUXEF8GPbo4GdgOHA59bDvfNDUkRERERNBsgDw77AX2x/G8D2CuDfgHdLurY835gSF/3psn2qpPdK2rtES18uaZGki8uAmxI3fb2k2ZKmlZS9zijq0yRdD5zYijccERERMVBlgDww7ALMrjfYfgL4LVUC3pskbQE8D7yhHPJG4MayPR44iWp5xo7AGyRtCJwDHGZ7AnABq85Ib2l7L9tf6toZScdJmiVp1oqnlq2jtxgRERExOOTX6wODgO6etyfgeuCDwBLgv4G/l7QpMMr24jIrfJvtBwBKrPQoYCkwBphRJpQ7gIdr176sUWdsTwGmAGw8cnSeAxgRERFDSgbIA8NC4NB6Q5kx3h6YA0wE7gVmAC8B3seqM87P1LZXUP25Clhoe1KDeyZqOiIiIqIbGSAPDNcAX5D0LtvfkdQBfAm40PYTkn4HvIMqCW9r4Mzy1ZPFwNaSJtm+uSy52Mn2wt50LFHTERERMdRkDfIA4CrO8O3A4ZLuAe4G/gL8RznkRuAPtp8q29uxcv1xo2s+CxwGnC5pHjAXeH3/vIOIiIiI9pGo6ehRoqYjIiKiHSVqOiIiIiKiSRkg9xNJlvTd2usNJD0q6apW9isiIiIiepYP6fWfPwNjJA2z/TTw98CDLe7TaiRtYPv5RvsTNR0RERHNapfI9swg96+rgc7/Uo4Cvt+5Q9Lukn5Z0vF+KWnn0n6spCsk/UzSPZK+WDvn/BLgsVDSZ2rtby0per+QdHbnLLWkzSRdIOn2cp+Da/eYKumnwPT+L0NERETE4JEBcv+6FDhS0ibAWODW2r5FwJ62xwOfBk6r7RsHHAG8BjhC0valfXJZTD4W2EvS2HLtrwP7234j1WPgOk0GrrW9G7APcIakzcq+ScAxtvddh+83IiIiYtDLEot+ZHu+pFFUs8f/02X3COAiSaOpUvQ2rO27xvYyAEm/AnYAfge8Q9JxVH9uI6mipf8GuNf2knLu94HjyvZ+wEGSPlpebwK8vGzPsP14d/0u9zgOoGOLrbs7JCIiIqJtZYDc/66kCvXYG9iq1n4qcJ3tt5dB9MzavtWS8SS9AvgosJvtP0m6kGrAqx7uLeBQ24tXaZT2oIckvURNR0RExFCWAXL/uwBYZnuBpL1r7SNY+aG9Y5u4zhZUg9plkl4G7E81qF4E7ChplO37qJZmdJoGnCDpBNuWNN72nN50Pkl6ERERMdRkDXI/s/2A7bO62fVF4POSbgI6mrjOPGAOsJBq0H1TaX8a+CDwM0m/AP4ALCunnUq1dGO+pDvL64iIiIjoQZL02oCk4baXSxLwNeAe219ZF9dOkl5ERES0oyTptb/3SZpLNbs8guqpFhERERHRB1mD3AbKbPE6mTGOiIiIGOoyQF5PJG0FXFNe/i3V0ykeLa93t/1sH687CrjK9pi17WNEREREZIC83tj+I1UACJJOAZbbPrOlnSJR0xERETFwDJSo6qxBbiFJEyRdL2m2pGmSRpb295V46HmSfihp09L+Mkk/Ku3zJL2+XKpD0jdKBPV0ScPK8a8skdWzJd0o6VWl/UJJX5Z0HXB6K957RERExECVAXLrCDgHOMz2BKpHt32u7LvC9m62XwvcBbyntJ8NXF/ad6X6UB7AaOBrtncBlgKHlvYpwAnl+h8FzqvdfyfgLbY/slrHpOMkzZI0a8VTy7rujoiIiGhrWWLROhsDY4AZ1dPZ6AAeLvvGSPr/wJbAcKrAD4B9gXcB2F5BFRryImCJ7bnlmNnAKEnDgdcDU8v1O+/ZaWq5xmqSpBcRERFDWQbIrSNgoe1J3ey7EDjE9jxJx1LFVPekazT1MKrfDiy1Pa7BOQ2jpiMiIiKGsgyQW+cZYGtJk2zfLGlDYCfbC4HNgYdL29GsjKS+BvgA8FVJHcBmjS5u+wlJSyQdbntqCREZWxL5mpao6YiIiBhqsga5df4KHAacLmkeMJdqSQTAp4BbgRnAoto5JwL7SFpAtZRilzXc42jgPeX6C4GD1133IyIiItpToqajR4majoiIiHbUU9R0BsjRI0lPAotb3Y9B7CXAY63uxCCV2q2d1K/vUru1k/r1XWq3dnpbvx1sb93djqxBjjVZ3Oinq1gzSbNSv75J7dZO6td3qd3aSf36LrVbO+uyflmDHBERERFRkwFyRERERERNBsixJlNa3YFBLvXru9Ru7aR+fZfarZ3Ur+9Su7WzzuqXD+lFRERERNRkBjkiIiIioiYD5IiIiIiImgyQoyFJ/yhpsaRfSzq51f0ZCCRdIOkRSXfW2l4saYake8r3F9X2fbLUb7Gkf6i1T5C0oOw7u0SBtzVJ20u6TtJdkhZKOrG0p35NkLSJpNskzSv1+0xpT/2aJKlD0hxJV5XXqV2TJN1X3vdcSbNKW+rXBElbSrpc0qLy79+k1K45knYu/811fj0h6aT1Uj/b+crXal9AB/AbYEdgI2Ae8OpW96vVX8CewK7AnbW2LwInl+2TgdPL9qtL3TYGXlHq2VH23QZMAgRcDezf6ve2Hmo3Eti1bG8O3F1qlPo1Vz8Bw8v2hlRx9K9L/XpVw38HLgGuKq9Tu+Zrdx/wki5tqV9ztbsIeG/Z3gjYMrXrUx07gN8DO6yP+mUGORrZHfi17XttPwtcChzc4j61nO0bgMe7NB9M9Q8g5fshtfZLbT9jewnwa2B3SSOBLWzf7Opv7Xdq57Qt2w/bvqNsPwncBWxL6tcUV5aXlxuWL5P6NUXSdsABwDdrzand2kn91kDSFlQTK98CsP2s7aWkdn3xZuA3tu9nPdQvA+RoZFvgd7XXD5S2WN3LbD8M1SAQeGlpb1TDbct21/YhQ9IoYDzVLGjq16SyRGAu8Agww3bq17yvAh8H/lprS+2aZ2C6pNmSjittqd+a7Qg8Cny7LO/5pqTNSO364kjg+2W73+uXAXI00t3anDwTsHca1XBI11bScOCHwEm2n+jp0G7ahnT9bK+wPQ7YjmpWZEwPh6d+haQDgUdsz272lG7ahmTtat5ge1dgf+BDkvbs4djUb6UNqJblnW97PPBnqiUBjaR23ZC0EXAQMHVNh3bT1qf6ZYAcjTwAbF97vR3wUIv6MtD9ofz6hvL9kdLeqIYPlO2u7W1P0oZUg+OLbV9RmlO/Xiq/op0J/COpXzPeABwk6T6q5WL7SvoeqV3TbD9Uvj8C/IhqGV7qt2YPAA+U3/YAXE41YE7temd/4A7bfyiv+71+GSBHI7cDoyW9ovzkdiRwZYv7NFBdCRxTto8BflJrP1LSxpJeAYwGbiu/DnpS0uvKp2jfVTunbZX3+i3gLttfru1K/ZogaWtJW5btYcBbgEWkfmtk+5O2t7M9iurfsmttv5PUrimSNpO0eec2sB9wJ6nfGtn+PfA7STuXpjcDvyK1662jWLm8AtZH/VrxScR8DY4v4K1UTxr4DTC51f0ZCF/lL+jDwHNUP5G+B9gKuAa4p3x/ce34yaV+i6l9YhaYSPU/mN8A51JSLdv5C3gj1a+05gNzy9dbU7+m6zcWmFPqdyfw6dKe+vWujnuz8ikWqV1zNduR6skA84CFnf8/SP2art84YFb5u/tj4EWpXa/qtynwR2BEra3f65eo6YiIiIiImiyxiIiIiIioyQA5IiIiIqImA+SIiIiIiJoMkCMiIiIiajJAjoiIiIioyQA5IiIiIqImA+SIiIiIiJr/BSBoN+tG3/mPAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "import matplotlib.pyplot as plt\n", "title_counts[:50].plot(kind='barh', figsize=(10, 8))\n", "plt.tight_layout()\n", "plt.title(\"50 most common job titles\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Before we start encoding the feature, we might want to look at whether these are informative at all. We can for example look at how frequently each is associated with a fully paid loan." ] }, { "cell_type": "code", "execution_count": 58, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(70672, 145)" ] }, "execution_count": 58, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# look at only the loans associated with a common job title\n", "loans_50_jobs = loans_paid[loans_paid.emp_title.isin(title_counts[:50].index)]\n", "loans_50_jobs.shape" ] }, { "cell_type": "code", "execution_count": 59, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array(['Supervisor ', 'Teacher', 'Manager', 'Police Officer',\n", " 'Truck Driver', 'Project Manager', 'Driver', 'supervisor',\n", " 'Registered Nurse', 'Operations Manager', 'Vice President',\n", " 'Program Manager', 'Executive Assistant', 'Sales', 'Supervisor',\n", " 'RN', 'Sales Manager', 'Software Engineer', 'Controller',\n", " 'Assistant Manager', 'Account Manager', 'manager',\n", " 'General Manager', 'President', 'Paralegal', 'Administrator',\n", " 'Director', 'Branch Manager', 'Mechanic', 'Technician',\n", " 'Accountant', 'Owner', 'Analyst', 'driver', 'teacher', 'Engineer',\n", " 'Store Manager', 'owner', 'Nurse', 'Administrative Assistant',\n", " 'Office Manager', 'Attorney', 'Server', 'sales', 'Foreman',\n", " 'Truck driver', 'Consultant', 'Electrician', 'Registered nurse',\n", " 'CEO'], dtype=object)" ] }, "execution_count": 59, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# ensure our subsetting did what we wanted it to:\n", "loans_50_jobs.emp_title.unique()" ] }, { "cell_type": "code", "execution_count": 60, "metadata": {}, "outputs": [], "source": [ "# group by the employment title and take the loan status column\n", "loan_status_by_title = loans_50_jobs.groupby('emp_title').loan_status\n", "# some panda kung-fu: for each title, compute rations of fully paid and charged off, take only fully paid ones\n", "paid_fraction = loan_status_by_title.value_counts(normalize=True).unstack('loan_status')['Fully Paid']\n", "# TODO seaborn will make this better!" ] }, { "cell_type": "code", "execution_count": 61, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Text(0.5, 0, 'Fraction fully paid')" ] }, "execution_count": 61, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAswAAAHgCAYAAACxVzvKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nOzdebxVVfnH8c9XcAARTCVFy0hzRkUBC6egyKw0NUfSEjNJTU1Li7QUNdNyyFnTUtQfijmPOaSgookCMoipmWLljANKoDJ8f3+sdWR7PecOcC6He8/zfr14cc/ae6+99sFePKye/TyyTQghhBBCCKG8ZWq9gBBCCCGEEJZmETCHEEIIIYTQiAiYQwghhBBCaEQEzCGEEEIIITQiAuYQQgghhBAaEQFzCCGEEEIIjehY6wWE9mu11VZzz549a72MEEIIoc176aWXAFhrrbVqvJL2a8KECTNsdy93LALmpZik3YAbgY1sPy2pN7Cm7Tvz8QHAh7YfqeEyK+rZsyfjx4+v9TJCCCGEEJok6cVKxyJgXroNBsYC+wDDgd5AX+DOfHwAMAtodsAsqaPteVVdZQVTX5pJz2F3LIlbhRBCCKEdmn7at2q9BCAC5qWWpC7ANsBA4FZJvwVOAjpJ2ha4BjgYmC9pP+Bw4N/AZUB34A3gANv/ljQCeAvYApgoaVXgXVLwvQbwc9vX5/seA+wFLA/cZPsESScDM2yfk885BXjN9rlL4KsIIYQQ6t6MO84GYLVvHVnjldSnCJiXXrsCd9l+VtJbQC/geKCv7cMAJHUCZtk+I3++DbjS9hWSfgCcm+cBWB8YZHt+DqB7ANsCGwK3AtdL2gFYD9gKEClQ3x74Myk15BxJy5B2vLcqt2hJQ4GhAB26lk0DCiGEEEILdey6Wq2XUNciYF56DQbOzj+Pyp+nNXFNf+A7+eergN8Xjl1ne37h8822FwBPSVo9j+2Qfz2RP3cB1rP9oKQ3JW0BrA48YfvNcguwfQlwCcDyPdZzE+sNIYQQQjOsvN1+tV5CXYuAeSmUUya+AvSSZKADYOCEFk5VDFj/1+DYB8VbFn4/1fYfy8z1J2AIKYXjsubcfNO1ujF+Kck9CiGEEEJYVFGHeem0Bym14nO2e9r+LPACsDawUuG89xp8foSULgGwL+mFwZa4G/hBzp9G0lqSPp2P3QTsCPTL54UQQghhCdlvv/3Yb7/YZa6V2GFeOg0GTmswdgOwEbCxpEnAqcBtpNzjXUgv/R0BXJZf3HsDOKAlN7V9j6SNgL9LglSBYz/gddsfShoNvNMgtSOEEEIIrWyDDTao9RLqmuxIM60lSWuQcpX7kdIkpgNH2n62hfMcCVxie3YLr5tlu4uknsDttntVOG8ZYCKwp+1/Nmfuvn37OuowhxBCCKEtkDTBdt9yxyIlo4aUtnFvAsbYXtf2xsCxpBfrWupIoHOF+3RY9FWCpI2B54D7SsHy4s4ZQgghhNBWREpGbQ0E5tq+uDRge5KS04FvkF7c+43ta3Nnv+HADFKZuQmklInDgTWB0ZJm2B4oaRZwFvB14GeStgJ+kG/zJ9ulChyfkIPh00iNUZYHLrC9jqQBOS3jFVITlY0be7hoXBJCCCFUxxu3/A6A7rv8osYrWXKWlqYlEAFzrZWC3oa+QwpINwdWAx6X9GA+tgWwCfAy8DCwje1zJf0UGGh7Rj5vReBJ28dL6kPKZ/4iqRLGOEkP2H6C8g4EZtruJ2l54GFJ9+RjWwG9bL+wGM8dQgghhBZYbvV1ar2EuhYB89JpW+Ca/HLda5IeIOU4vws8Zvu/APnlv56Ur4Yxn/SiYGm+m2z/L193I7AdC+stN7QDsJmkPfLnbqSGJh/m+1cMlqNxSQghhFB93b60Z62XUNciYK6taaQScg2pzFhJsX7yfCr/Gb5fqGbR2HzlCDjc9sfKx+WUkIb1nD8mGpeEEEIIob2JgLm27gd+K+kg25cCSOoHvA3sLekKYBVge+AYUhvrSko1mWeUOfYgMELSaaRgeDfge43MdTdwiKT7bc+VtD7wUsseLRqXhBBCCNWy++67A3DDDTc0cWZoDREw15BtS9oNOFvSMOB9clk5UlvqyaSX/n5u+1VJjQXMlwB/lfSK7YEN7jNR0gjgsTz0p0bylyF19esJTMyVPN4Adm3p84UQQgihOvr371/rJdS1qMMcWk3UYQ4hhBBCW1F3dZglHSdpmqQpkiZJ+mIT52+Xz58kqb+kby6ptZZZy/y8jtKvYYsx1yPVXFsIIYQQQj1qdykZkvoDOwFb2v5A0mrAck1cti9whu3LJQ0B+gJ3VnFNHVrQTnqO7d7VuK/trasxTyWSOtqeV+l41GEOIYQQquP1G04C4NO7H1/jlbS+pan+ckl73GHuAcyw/QGA7Rm2XwaQ9FVJT0iaKukySctL+iGwF3C8pGuAk0gv3E2StHc+d+XcTORNSd/Pc10laZCknpIekjQx/9o6Hx8gabSkq4GpkjpIOl3S43nn+0cteShJ0yWdmO8xtZTPLKm7pHvz+B8lvZj/kUBuXlJayxhJ10t6WtLInJuMpD6SHpA0QdLdknrk8XUl3ZXHHyrcb4Sks3IDk98t1p9UCCGEEJplhc9tzgqf27zWy6hb7TFgvgf4rKRnJV0o6csAklYARgB7296UtLt+iO0/AbcCx9geDBwPXGu7t+1ryc1BSM1CnifVLwb4EvAo8DrwNdtbAnsD5xbWshVwXG55/VEzEFJN5YMkfb7M+js1SMnYu3BsRr7PRcDReewE4P48fhOwdoXvZQvSy4QbA+sA20haFjgP2MN2H+Ay4JR8/iWk0nJ98r0uLMy1PjDI9s8q3CuEEEIIVdS17y507btLrZdRt9pdSobtWbmz3Xak1tPX5jzgJ4AXbD+bT70C+DFQsUV09hCprNuLpEB1qKS1gLfyvboB50vqTaqLvH7h2mKTj0rNQBo2AWksJePG/PsEUjdASE1JdsvPfpektytcW67hyTukboP35g3nDsArkroAWwPX5XFILbJLrquUYhKNS0IIIYTQ3rS7gBkgB3NjgDGSpgL7A5MWcboHSYH12sBxpOB0D1IgDXAU8BqpjfUypNJwJcUmH2WbgbRQqWlJsWFJc5uSlGt4ImCa7Y/VqpHUFXinkcC9YvOSaFwSQgghVN9rfzkBgNX3OrHGK6lP7S5glrQBsMD2P/NQb9Lu8NNAT0lfsP0cqXHHA2WmKDUAAcD2f0ovDtp+XtJYUorCYfmUbsB/bS+QtD9pl7acss1ASu2qF8NYUg727yTtAHyqBdc+A3SX1N/233OKxvq2p0l6QdKetq/L+c6b2Z7ckoVF45IQQgihOi5c+0UADj00/l6thXYXMJMafpwnaWVgHvAcMNT2+5IOIKUZdAQeBy4uc/1oYFhOWzg15zGPY2Eg/BBwKilQhZTbe4OkPfO1lQLg5jYD6ZTvXXKX7cZKy50IXJNznR8AXiEF/U2y/WFOETk3p5Z0JKWoTCNVDrlI0q+AZYFRpEYqIYQQQljCDj300Fovoa5F45I2TtLywHzb83JJvYuqVZZucUXjkhBCCCG0Faq3xiXVlkuyfb3B2JG5Cse3tRjNRRrc4xlJkyU9nFNLmmNt4HFJk0kVOg4qzHmSpEFl7jVA0u2LsdZjF/XaEEIIIbTcoEGDGDToE3+lhyUkdpibIddM/pLtAwpjj5JK0T1U+coW3WMMcLTt8bnSxE62v93gnJY0QGnsXgPyvXZaxOtn2e7S1HnL91jPPfZvqghJCCGEEJry3qS7AFip9441XsniWxobk0DsMFfD9cBOOf0BST2BNYGxkoZIOj+Pry7pprxLPFkLm5jsJ+mxXFf5j5IqvRhY8iDwhXztrLxTPA7oX26u/GuEpCeVmpocla8dUSpjJ2nH3LRkLAtL0iFpRaUmLo8rNXXZJY8PkXRjbl7yT0m/z+OnsbBW9MjqfL0hhBBCaMxKvXdsF8FyWxUBczPYfhN4DCj9l7oPqblJw+35c4EHbG8ObAlMk7QRqaHJNjm3eD7phbrG7AxMzT+vCDxp+4vAmxXm6g2sZbtXbspyeXGy3LTl0jzvdsAahcPHkRqf9CPVrT5d0or5WO98v01J3Q8/m19AnJMbu3ziOSQNlTRe0vj5s2c28ZghhBBCCEu/CJib7xpSoEz+/Zoy53yF1NwE2/NtzwS+CvQh5RlPyp/XqXCPkfmcbVjYyW8+cEP+udJczwPrSDpP0o7Auw3m3ZDUtOWfOcj/v8KxHVhYFWQMsAILuwXeZ3um7feBp4DPVVj3R2xfYruv7b4dOndr6vQQQgghNMOrVw/j1asX+5WpsIjaY1m51nIzcJakLYFOtic28zoBV9j+ZTPO3dd2w7IS7xfylivOJWlz4OukJit7AT9ocEqlZHUBu9t+psF8X6R8s5MQQgghLGFdNo0X/mopAqBmym2wxwCXUX53GeA+4BDg7JynvGIeu0XSH2y/LmkVYCXbLy7CMsrORar9/KHtGyT9CxjR4Lqngc9LWtf2v4DBhWN3A4dLOty2JW1h+4km1jFX0rK25zZ2UjQuCSGEEKol/j6tpUjJaJlrSC2wR1U4/hNgoFI77gnAJrafAn4F3CNpCnAv0GNRbt7IXGuR2oBPIgXLv2xw3fvAUOCO/NJfMVg/mdSYZIqkJ/PnplySz4+X/kIIIYQlYO7cucyd2+g+VWhF7b6snKT5pBfoOgL/APa3Pbu2q2qcpOnAf2xvVxibBHS03atmC2uhaFwSQgghVMeAAQMAGDNmTE3X0Z41VlauHlIy5pQ63+Ud0YOBs0oHq1HbuFr1kRtYKVel+E+utLHUaaXnDiGEEEIDP/zhD2u9hLpWDzvMHzXZkHQwsBnwF+AE4BVS6bQtSdUt+gLzgJ/aHi2pMynFYUPS7nRP4Me5ucgsUuD9deBnpAoZOwOdgEeAH+Wc4DHAE6TqFt2B75NSJjYllab7VZk1TyelPXxo+wxJJ5HylL9nu1euA30VKUca4DDbj+SGJMOBGUAvUlrIfnkdx1dYXz/gz3n+scA38j06AKcBA4DlgQts/zHf46PvzvbGlb77aFwSQggh1KeltTlJY6JxCSCpI/ANFtY33go4Lgd8PwbINYwHA1fk2sWHAm/b3oyU29unMOVH9ZFtjwXOt90vp0x0Aopd9D60vT1wMXBLvl8vYIikVSss+XoWNhjZGbitcOx14Gu2tyTVST63cGwL4EhgY1LJuW3yeKX1XQ4cbLs/qRJGyYHAzFyfuR9wkKTPl/nuQgghhNDKFsx9nwVz36/1MupWPQTMnXL+73jg36TdVIDHbL+Qf96WtGOL7adJL8Wtn8dH5fEngSmFeYv1kSG97Dcuv/D3FWCTwrFb8+9TgWm2X7H9Aal+8mcrrPst4G1J+5B2t4t518sCl+Z7XUcKjkses/1f2wuASaRd8bLrk7QyqWLHI/mcqwvz7AB8P39344BVgfUK93iBMqJxSQghhFB9r183nNevG17rZdStusphLpEEKQXho6EK11Yah0J95LwbfSHQN+ccDyc1ACkp1TNewMdrGy+g8T+Da4ELgCENxo8CXiNV7FgGKP6T8xO1kxtZX2PPJ+Bw23d/bDClZPyv7BWkxiWkdBKW77Fe+873CSGEEJaQlbb4Zq2XUNfqIWBujgdJLabvl7Q+qdPdM6Sc3r2A0ZI2JuUdl1MKjmdI6gLsQUqpWFw3kcrG3Q2sWRjvBvzX9gJJ+wMdmpin7Ppsvy3pPUlfsv0oCzsZku95iKT7bc/N38tLLVl81GEOIYQQqiX+Pq2lCJiTC4GLc7rCPGCI7Q8kXUjKZ55CenFvCvCJPAPb70i6lJRyMR14vBqLsv0e8Dv4aFe8uN4bJO0JjKaRHd9mrO9AUnrH/0itsUvP9ydSOsdEpZu/Aey6WA8UQgghhEUyc2b667lbt241Xkl9avdVMhZHrhSxrO33Ja1L6rS3vu0Pa7y0qpHUxfas/PMwoIftn1Rj7qjDHEIIIVRH1GFuffVeh3lxdCalYyxLyuk9pBbBcqH5yrKkHfArgLNzSkZf4Pu2j1jE6b8l6Zek/xY6Ats1cX4IIYQQlrAjjljUv+ZDNcQOcxvQoJb0p0nVLB62fUIj13S0PW9R79PM80X6b2hBueOxwxxCCCGEtiJ2mNsR269LGgo8nqtdfBk42vZO+fOapNzjGZJ+Qqr9vHa+/EjbD+cX/84jNWoxcCKp1nKpBN802/tK+inwg3ztn2yfnZum/JWUO92flNf8Yrm1Tn1pJj2H3VHNxw8hhBDqUqlUa4fOS28Oc1tsVtJcETC3Qbafl7QM8Okyh/sA29qeI+lq4A+2x0pam1T5YiPg16SmJJsCSPqU7RskHVZoI94HOAD4IikdZZykB4C3gQ2AA2wf2sqPGkIIIQTgjZtPBWCN755W45XUpwiY265KNZRvtT0n/zwI2LhQYaOrpJXy+Ecl5Gy/XWaebYGbbP8PQNKNpPzmW4EXcxm6Ty4q7X4PBejQtXuLHiiEEEII5XXdardaL6GuRcDcBklah9SU5HXSjnFRscTcMkD/QgBdul6kVIxGb9PIsWhcEkIIISxBnb/wxVovoa5FwNzGSOpOyks+37Yb1Gdu6B7gMOD0fG1v25MK40fm8U/lXea5kpa1PZfUzGWEpNNIwfNuwPdastZoXBJCCCFUx6uvvgrAGmusUeOV1Kdlar2A0CydJE2SNA34GyngPbEZ1x0B9JU0RdJTwMF5/DfApyQ9KWkyMDCPXwJMkTTS9kRgBPAYMI700t8T1XukEEIIITTXPvvswz777NP0iaFVRFm50GqirFwIIYRQHXfddRcAO+64Y41X0n41VlYudpiXIElrSBol6V+SnpJ0p6T1W/mePSU9mX/uLembiznfsdVZWQghhBCaa8cdd4xguYYih3kJyS/a3QRcYXufPNYbWB14dgktozep9vKdizHHscBvm3Ni1GEOIYQQqmPeu28A0HEprkDVnuswxw7zkjMQmGv74tJAfgFvrKTTcz7xVEl7A0gaIGmMpOslPS1pZA66kXRa3qGeIumMPDZC0h6luSXNKt5c0nLAScDeOR96b0lbSXpE0hP59w3yuUMk3SjpLkn/lPT70n1ZmE89slW/rRBCCCF8ZMbtZzLj9jNrvYy6FTvMS04vYEKZ8e+Qdn43B1YjdfB7MB/bAtgEeBl4GNgmv7y3G7BhrpKxcnNubvtDSccDfW0fBiCpK7C97XmSBpF2jnfPl/TO9/8AeEbSebaHFZublBN1mEMIIYTq67Z1vPBXSxEw1962wDW25wOv5W56/YB3gcds/xcgt6zuCTwKvA/8SdIdwO2Lce9uwBWS1iPVZV62cOw+2zPzvZ8CPgf8p6kJow5zCCGEUH2delbcqwpLQKRkLDnTSG2rG2qskPIHhZ/nAx1tzwO2Am4AdgXuysfnkf88c+rGcs1Y08nAaNu9gJ2BFRq7dzPmCyGEEEIrmPvOq8x959VaL6NuRRC05NwP/FbSQbYvBZDUD3iblFd8BbAKsD1wDLBhuUkkdQE6275T0qPAc/nQdFJA/hdgFz6+W1zyHrBS4XM34KX885BmPkexuUmjonFJCCGEUB0DBgwAYMyYMTVdR72KgHkJyfnGuwFnSxpGSquYTuq21wWYTEqL+LntVyWVDZhJAe8tklYg7U4flccvzeOPAfdRvn31aGBYTu84Ffg9KSXjp6SAvjlKzU0m2t63mdeEEEIIYTGceGJz+pWF1hKNS9qgHHjfCGxk++lFnGMEcLvt61t43RDgHtsvN3VuNC4JIYQQQlsRjUvan8HAWKAWr8wOAdaswX1DCCGEuvXMM8/wzDPP1HoZdStSMtqYnMO8Damu863AcEkDgOHADBaWr9svp4EcT3qhrxPwCPAjF/5vBUlfBQ6zvVv+/DXgEGBP4M+kRicGLiNVyegLjJQ0B+hve06ltUbjkhBCCKE6Xr16GABrfPe0Gq/kk9pzw5KSCJjbnl2Bu2w/K+ktSVvm8U/UbCbtQp9v+yQASVcBOwG3Fea7H7hAUnfbbwAHAJeT6jCvlStoIGll2+9IOgw42nbkWoQQQghLyMrb71/rJdS1SMloewYDo/LPo/JnyDWbbS8ASjWbAQZKGidpKvAVUlD9kbzbfBWwX26C0h/4K/A8sI6k8yTtSKoL3SRJQyWNlzR+/uyZi/yQIYQQQlhohc9sxAqf2ajWy6hbscPchkhalRT09pJkoAMpXeJOytRNzpU0LiR19/uPpOF8vNZyyeWkXef3getyree3JW0OfB34MbAX8IOm1hiNS0IIIYTq+/CN6QAs171nTddRryJgblv2AK60/aPSQO4MuG2F80vB8Yyc+7wH8ImqGLZflvQy8Cvga3ne1YAPbd8g6V/AiHx6w1rOFUUd5hBCCKE6PqrD/OcxNV1HvYqAuW0ZDDTM9r+B9JLevxqenHOOLwWmkmo+P97I3COB7rafyp/XAi6XVErb+WX+fQRwcXNe+gshhBBCdZx++um1XkJdizrMAQBJ5wNP2P5zteaMOswhhBBCaCuiDnMbImm+pEmFX8MWY65HmnneBGAz4P8W9V4hhBBCaD2TJk1i0qRJtV5G3YqUjKXPHNu9qzGR7a2beV6fRZlfUsf8gmAIIYQQWtGRRx4JwJgxY2q7kDoVKRlLGUmzbHcpMz4duILUhGRZYE/bT0vqDlwNrErKUd4R6GN7RmmuJhqb9AHOArrk40NsvyJpXeACoDswGzgo328E8Bap7vNE2z+r9CzL91jPPfY/e7G/kxBCCKHeffja8wAst/o6NV1He25SEikZbUunBikZexeOzbC9JXARcHQeOwG4P4/fBKxdYd4tgCOBjYF1gG0kLQucB+yRd5kvA07J518CHJ7HjyaVpytZHxjUWLAcQgghhOpZbvV1ah4s17NIyVj6NJaScWP+fQLwnfzztsBuALbvkvR2hWsfs/1fAEmlxibvkHac75UEqa7zK7kE3dbAdXkcYPnCXNfZnl/uJpKGAkMBOnTtXvkpQwghhNBsH7zyLADL91i/xiupTxEwty2l5iTzWfhnpwrnVrq2eL2Aabb7F0+U1BV4p5HA/X+VbhKNS0IIIYTqe3v0ZQCs8d2G1WXDkhABc9s3ltSF73eSdgA+1YJrnwG6S+pv++85RWN929MkvSBpT9vXKW0zb2Z7cksWFo1LQgghhOp4cr/PAdCrV68ar6Q+RQ7z0qdhDnNT/5Q8EdhB0kTgG8ArpG58TbL9Ian73+8kTQYmkVIxAPYFDszj04BdFuFZQgghhFAFvXr1imC5hqJKRhsnaXlgvu15kvoDF1WrLN3iisYlIYQQQnU88khqrbD11s2qGBsWQbutklHNJh8tuGdPSd8tfO4r6dwqzn+OpJcKLambsjbweN4JPhf4n6SVG5n/SEmdF2N9AyTF/1pDCCGEJejYY4/l2GOPrfUy6lab3mGuVLO4le85ADja9k6tMPcywHTgZWCY7TGtcI/pQF/bMxbx+uHALNtnNHVu1GEOIYQQqmPum/8FYNlVP7PE7tmeay6X0253mMuR1E3SM5I2yJ+vkXRQ/vkYSY9LmiLpxMI1389jkyVdlcdGSNqjcM6s/ONpwHZ5R/uovON6u6RlJE0v7u5Kek7S6pK6S7oh3/txSdtUWP5A4ElSneXBhXm+XNhFf0LSSpJ6SHowjz0pabt87nRJq0laUdId+ZmelLS3pCOANYHRkkbn8y+SNF7StAbfyXRJJ0qaKGmqpA0l9QQOBo7K991u0f6UQgghhNASy676mSUaLIePa+tVMjrlmsIlp9q+VtJhwAhJ5wCfsn1priCxHrAVqZzarZK2B94EjgO2yd3xVmninsMo7DDnHWdsL5B0C6km8uWSvghMt/2apKuBP9geK2lt4G5gozJzDwauAW4BfitpWdtzSY1Dfmz74Vwj+X1SreO7bZ8iqQPQMM1iR+Bl29/K6+xme6aknwIDCzvMx9l+K89xn6TNbE/Jx2bY3lLSofmZfyjpYpq5wxxCCCGE6nj/31MBWGHtTWu8kvrU1neY59juXfh1LYDte4GppNbOP8zn7pB/PQFMBDYkBdBfAa4vBZC231qM9VwLlDrz7ZM/AwwCzs/B/a1AV0krFS+UtBzwTeBm2+8C4/J6AR4Gzso7xCvbnkdqg31ATpHY1HbDyhhTgUGSfidpO9szK6x5r1xh4wlgE1InwJJio5SezfkCJA3NO9bj58+udMsQQgghtMQ7Y0fyztiRtV5G3WrrO8xl5VzgjYA5wCrAf0m7yqfa/mODc48AyiVyzyP/g0KSgOWaceu/A1+Q1B3YFfhNHl8G6G97TiPX7gh0A6am29EZmA3cYfs0SXeQAupHJQ2y/WDeIf8WcJWk021fWZrM9rOS+uRrTpV0j+2TGjz750m71/1svy1pBLBC4ZRyjVIaFY1LQgghhOpb9ZtH1noJda1dBszAUcA/gGOBy5TKrd0NnCxppO1ZktYC5gL3ATdJ+oPtNyWtkneZpwN9gL+QahAvm+d+D1iJMmxb0k3AWcA/bL+ZD90DHAacDiCpt+1JDS4fDPzQ9jX5nBWBF3JFix62p5KC6f7AhpLmAC/ldJMVgS2BjwJmSWsCb9n+v5x/PaTB+mcAXUld+2ZKWp1Ux3lME9/te/m6JkXjkhBCCCG0B209YG6Yw3wXcBkpDWMr2+9JehD4le0TJG0E/D3v4M4C9std7U4BHpA0n5SaMAS4FLhF0mOkoLrUDnoKME+pjNuIfH7RtaR0iSGFsSOACyRNIX3nD5JengMgB8VfB35UGrP9P0ljgZ2BbSUNJO30PgX8lZTycYykuflZvt9gHZsCp0taQPqHwSF5/BLgr5JesT1Q0hOkxiTPk1I/mnIbcL2kXYDDbT/UjGtCCCGEsBj+9re/ATBo0KAar6Q+temycmHpFo1LQgghhOoYMGAAAGPGjKnpOtoz1VNZuZbSwuYnT0q6Ti1o6qHFaFqiRhqISBoj6d85d7o0drMWlrYLIYQQQh256qqruOqqq2q9jLpV9zvMKjQ/kTQSmGD7rMLxDrbnt8J9p1OhgYikMaSXFQ/NpehWJuVgb7KkG7U0Jgf0sr2g3PFoXPt7jSgAACAASURBVBJCCCG0TfXWtARih7klHiJVuRggaXSunzxV0gqSLs8NPJ7I+cSlNtG3559XlHSZUmOSJ3KOL5I6SDojXztF0uEq00CkjFGkPGWA77CwxBuSuki6TwubipTu1VPSPyRdqtSI5B5JnfKxg/LaJis1Uemcx9eV9Gg+dlJxF1tlGr0U7nEhqTzfZ6vz1YcQQgihkjnPT2DO8xNqvYy6FQFzJqkjqUrE1Dy0Fampx8bAjwFsb0qqZnGFpBUaTHEccL/tfqSOfafn6hVDgc8DW9jeDBhp+1xS++uBtgdWWNJ9wPZKDUWKNZ0hNS7ZzfaW+V5nFtI31gMusL0J8A6wex6/0XY/25uTKogcmMfPAc7J63658H0UG730BvrkMnYAGwBX2t7C9osNvseowxxCCCFU2cxHr2Pmo9fVehl1q61XyaiGYqWNh4A/A1sDj9l+IY9vC5wHYPtpSS8C6zeYZwfg25KOzp9XANYmNS25ODcbaUljlPnAWFIjlE62pxdTmkmdALcHFgBrAavnYy8UStYVG470kvQbYGWgCynFA6A/qWY0wNVAqYNfsdEL+Zr1gH8DL9p+tNyiow5zCCGEUH3dv/2LWi+hrkXAnLsFFgdyYPq/4lAz5hGwu+1nGswlyjdGaY5RwE3A8Abj+wLdgT625+Z86NKO9weF8+YDnfLPI4BdbU+WNAQY0MS9KzV66cnHv5uKog5zCCGEENqDSMlongdJQSqS1iftHD/T4Jy7gcNLqRGStsjj9wAH55QPJK2Sxys2QCl4CDgVuKbBeDfg9RwsDwQ+14xnWAl4RdKypWfJHmVh2sY+hfG7gR9IKr0QuZakTzfjPiGEEEKosttuu43bbrut1suoWxEwN8+FQAdJU0m5xENsl3ZyS7vHJ5O6AU6R9GT+DPAnUhrDlNzs5Lt5vNRApNJLfzg5o0wljZFAX0njScHv0814hl8D44B7G5x/JPDT3KClBzAz3/seUorG3/NzX0/TAX4IIYQQWsGZZ57JmWeeWetl1K26Lyu3OJQ6A84kvSz3D2B/27MrnDuEVEbuMEkHA7NtX1nu3Bbcvxspt3qbPPQwqfvezHz8dOCbwJ3A74HbgeVInQd/SQrePySlpVjSPsBg27sszrpKonFJCCGEUB0zZqS9s9VWW63GK2m/oqxcK5D0bVKe7062e5ECz4MbvyqxffHiBsvZn4Hnba9re13gBdKOdsmPgC1tHwN8FXg6V7Z4yPY3bb8D9AEmKbXtPhT4WXNvXkozCSGEEELrWm211SJYrqEIeBaR7Vslzbb9SB56CNgs5yhfBqwDzAaG2p5SvFbScGCW7TMkfQG4mPQS33xgT9v/knQMsBewPHCT7RMazPEFUrC7d2H4JOA5SesCfwBWBMZJuoZUGq9UEaQ/aUe8r+2HJJ0JHE3KjT4B+J6k7nlda+e5j7T9cF77mqTqGzNYmGLyCVNfmknPYXc09VWGEEIIoQmzn0nhRucNtm71e9Vj05KmRMBcBYUazncBJwJP2N5V0leAK0l1jCsZCZxm+6Zc23mZBjWQBdwqaXvbDxau2xiYVOxCaHt+Dog3sf1tpS6GvfMaXyOnhOTPpbVvQqohvY3tGYWXEs8B/pA7Da5Neglwo3ysD7Ct7TmL8n2FEEIIoWXenXArsGQC5vBJETAvnnI1nMeRq07Yvl/SqjnX+BMkrQSsZfumfP77ebxSDeRiwFypXF1Ly9h9Bbi+9GJhoU70IGDjQu3nrnm9ALdWCpYlDSU1a6FD1+4tWEYIIYQQKvn07r+u9RLqWgTMi6dcDedyNZsrBbCV6juXrYHcwDRgC0nL2F6Q770MUOrk11yVAuxlgP4NA+MyNao/JhqXhBBCCNW3zPIr1noJdS0C5uor1Ww+WdIAYIbtd8vF0Xn8v5J2tX2zpOWBDqT0h5MljbQ9S9JawFzbrxeufU7SE8CvSLnL5J8n2n6uBeu9D7hJ0h9svylplbzLfA9wGHA6gKTehQ6CzRKNS0IIIYTquPbaawHYe++9mzgztIaoklF9w0k1kqcApwH7N3H+94Aj8vmPAGu0oAbygcD6kp6T9C9Su+4DW7JY29OAU4AHcp3os/KhI0rPIekpmlkBJIQQQgjVd9FFF3HRRRfVehl1K+owh1YTdZhDCCGE6pg9O7V56Ny5c41X0n61mzrMkj4j6RZJ/5T0L0nnSFpuCdx3iKQ1C5//JGnjVrpXT0mWdHJhbDVJcyWd3xr3DCGEEMLSrXPnzhEs11CbyWHOL9PdCFxkexdJHUgvl50CHFOF+TsUS7Q1MAR4ktTRD9s/XNz7NeF5YCdSO2uAPUkv+S1VJHW0Pa/S8ajDHEIIIVTHrGmjAeiyycBWu0fUX66sLe0wfwV43/blkGoOA0cBP5DUOe8C3yLpLknPSPqo0Yek/SQ9JmmSpD/mYBtJsySdJGkc0F/S8ZIel/SkpEuU7AH0BUbm6ztJGiOpb55jsKSp+ZrfFe45S9IpkiZLelTS6nl8z3zuZEnFMnFFc4B/lO5Bak7yl8LcO0saJ+kJSX8rzD1c0mV5fc9LOqJwzc2SJkialku/lcYPlPRsvubS0i62pO6Sbsjfx+OStinc4xJJ95BqTIcQQgihlc2afDezJt9d62XUrbYUMG8CTCgO2H4X+DfwhTy0FalCRW9gT0l9JW1ECji3ySXg5udzIHXCe9L2F22PBc633S+3uu5Eant9PTAe2Nd272KZtZym8TtSMN8b6Cdp18Lcj9renFQ546A8fjzw9Tz+7UaedxSwj6TP5DW/XDg2FviS7S3yeT8vHNsQ+Hr+Lk6QtGwe/4HtPqTg/4hcH3pN0i72l4Cv5WtLSo1L+pHqShdbbvcBdrFdsctfCCGEEKpn9b1/w+p7/6bWy6hbbSYlg+Y16rjX9psAkm4EtgXmkQK8x3Npt05AqTzbfOCGwlwDJf0c6AysQkqDuK2RNfUDxth+I99zJLA9cDPwIXB7Pm8CKSAFeBgYIekvpBSTSu4CTgZeA65tcOwzwLWSegDLAS8Ujt1h+wPgA0mvA6sD/yUFybvlcz5LaoSyBvBAqVmJpOtIlTYgGpeEEEIISw11aEshW/vTlr79aeQOeiWSupKCv3+RguKGAbVJAfUVtn9ZZs73S3nLSm2pLyS1j/6PpOHACk2sqVLjEUh1k0vrmU/+rm0fLOmLwLeASbm+8ZsNL7b9oaQJwM9Iu+s7Fw6fB5xl+1alWs/DC8c+KPw8H+iYzxlEakQyW9KY/GyNrT8al4QQQghLiVlT/wZAl00H1Xgl9aktBcz3AadJ+r7tK3Me8pnAiBwEAnxN0iqkHOBdgR8As4FblBpzvJ6Pr2T7xQbzl4LjGZK6AHuQ6h8DvEf5OsjjgHMkrQa8DQwmBbMVSVrX9jhgnKSdSQH/JwLm7EzSDvCb+njjk27AS/nnpuo8l85/O39PG5JSMAAeA/4g6VOkZ9wdmJqPReOSEEIIYSkxYMDpAIw57Q81Xkl9ajMBs23nlIILJf2atAN6J3Bs4bSxwFWknOarbY8HkPQr4B6l1tFzgR8DLzaY/x1Jl5ICxunA44XDI4CLJc0B+heueUXSL4HRpN3aO23f0sSjnC5pvXz+fcDkRp55GuWrYwwHrpP0EvAo8Pkm7nkXcLBSc5Rn8jXYfknSb0mB/8vAU8DMfM0RwAX5mo6kPOxoXhJCCCHUwJgxY2q9hLrWbhqXSBpCSqc4rNZraUskdcnttzsCNwGX2b6pGnNH45IQQgghtBVqL41Lqk3S/FwqbrKkiZK2buX79ZT0ZDPOG67UvOQLhbGj8ljZP8jFMFzSJFKd6RdILyyGEEIIYSly6aWXcumll9Z6GXWrzaRkNMX2CFLqREvMyaXmkPR14FTgy8UT1HhDk9Y0FdgHKNWQ2YOUMlFVto9enOsb+36icUkIIYRQHa+NugCAU/61ZhNnLppoWtK4ut5hbqAr6cU9JA2QNFrS1eSX4Bpp/FGpQcnqkm7K45MLu9cdcoOQaZLukdSpwnpuBnbJc61Dyi1+o3DfiySNz/OcWBifLunEvGM+Nb/kh6StJD2i1OzkEUkb5PHOkv4iaYqka5UaopSasuwg6e95ruvyy5ClexwvaSypC2EIIYQQWtHq+5zC6vucUutl1K16D5g75ZSMp0mNOU4uHNsKOM72xvnzJxp/5PFKDUrOJVW42BzYkoUv760HXGB7E+AdGpTKK3gX+I+kXqTqGw1rMR+X82w2A74sabPCsRm2twQuAko7yE8D2+dmJ8cDv83jh5IqaGyWn78PQK788StgUJ5rPPDTwj3et72t7VHFRUkamgP58fNnzySEEEIIoa1rNykZi6iYktEfuDIHqACP2S42BCnX+ONNKjco+QrwffiojffMXL7thUJ5tglAz0bWN4qUlvF14KvAAYVje+Wd7o5AD2BjYEo+VmqIMgH4Tv65G3BFrtBhoNQBcFtSVz9sP5mrYkAqPbcx8HAuabcc8PfC/RsG8OQ5og5zCCGEUGXvTUwpjittGakTtVDvAfNHbP8976qW2tN91JyjkcYfUKFBSSMaNhaplJIBqcvg6cB42++WajFL+jxp57if7bcljeDjTVZK9yiu52RgtO3dJPUExpQer8K9ReqcOLjC8YrNS0IIIYRQXbOfewyIgLlWImDOcq5vB8o3EanU+KMx9wGHAGfnJisrtnRNtudI+gXwbINDXUkB68ycM/0NFgbAlRSbnQwpjI8F9gJGS9oY2DSPP0qqw/wF289J6gx8xnbDtVQUjUtCCCGEKom/T2sqcphTDvMkUorB/hUqPtxFajE9hbRT+2gz5v4JMFDSVFJqxCaLskDbo2xPbDA2GXiClBd9GfBwM6b6PXCqpIdJ/zAouRDonp/tF6S0jpm23yAF1tfkY48CGy7KM4QQQgghtGXtpnHJkpbzmW8ENrL9dJnjY4CjS90GmzFfX+D7to9o4rxHbFesFy3pWNu/rXS8zPkdSPnMBwP3AncA69v+sLlzVBKNS0IIIYTqOOeccwD4yU9+UuOVtF/RuKR1DCalM+xTjclsj28qWM7nNdVc5dhyg0rK/Xl3Jj3H74G/AIc0N1jOwXYIIYQQWtl9993HfffdV+tl1K3YYV4EuR7xM8BA4FbbG+Z6ypeTKkv8g1T94se2x0uaBVxAenHwbVJQ+3tgbeBI27fmFwuPtr2TpOH52Dr597Ntn5vvPct2F0k9SGkkXUm56IcA3wKOIdWOngYcB/wVGA30B3YFhgH9SC8bXm/7BElHAGfkZ5phe6CkwXmdAu6w/YvS/YGzSJU7fmZ7bKXvafke67nH/mcv4rccQgghhNYWDUsWih3m6tsVuCu/APeWpC1JAevsXM/4FHI942xFYEyu4/weqXvf14DdgJMq3GNDUlC6FXCCpGUbHP8ucHcui7c5MMn2MHKpPNv75vM2AK60vYXtFylTvzkH4y8DA3OwvCbwO1JpvN5AP0m7Fp7lSdtfbCxYDiGEEEJoLyJgXjSDSTWSyb8PBrYH/g/A9hQW1kSGVKv5rvzzVFJDk7n5554V7nGH7Q9szwBeB1ZvcPxx4IC8G72p7fcqzPOi7eJLintJmkh6aXAT0o54Q/1IAf4btucBI/PzQSpVd0OFe0XjkhBCCKEVzBx3IzPH3dj0iaFVRFm5Fsod/r4C9JJkUsUJkwLQSvktxVrNC8h1km0vkFTpz6BhveaPnWf7QUnbk9IwrpJ0uu0ry8xTrCfdVP3mj06tsCZIHf7KVRIprSsal4QQQghV9uHLn6gvEJagCJhbbg9SisOPSgOSHgAmAvuS6hn3IqU8tBpJnwNesn2ppBVJ7bevBOZKWjbvYDfUWP3m94CVgBnAOOCc3MjlbdIO+nktXWPUYQ4hhBCqJP4+rakImFtuMHBag7EbgC1IdZ2nAJOAx1p5HQOAYyTNBWaR23CTdnen5LSL44oX2J4sqVS/+Xk+Xr/5EuCvkl7Jecy/JL0sKOBO27e06tOEEEIIISylokpGaDVRhzmEEEKojtNOS3t1w4YNq/FK2q+oklFjknaT5NxWu5br2DW3v17U63tK+m411xRCCCGEpk2aNIlJkybVehl1KwLmJaOqTU4Ww66Ur4rRXD1J5exCCCGEsASNGjWKUaNGNX1iaBWRktHKKjQ56UCqc/x1UmWNS22fJ6kfcA6p1vEHwFeBucBFQF9gHvBT26MlDQH62j4s3+d24AzbY3JzkXOAnYA5wC7AusDtwMz8a3dStY+hwHLAc8D3bM/O1TPezfdcA/i57eslPQpsBLwAXGH7D409ezQuCSGEEJYu0aikskjJqK1yTU6GAp8HtsiNTkZKWo7Uue8ntjcndQWcA/wYwPampJ3qKySVKwVXtCLwaJ7nQeAg248AtwLH5MYm/wJutN0vn/cP4MDCHD2AbUlBd+klx2HAQ/n6RoPlEEIIIVTPOw9fwzsPX1PrZdStqJLR+gYDpW3WUpOTdYCLc1MQbL8laVPgFduP57F3ASRtSy7pZvtpSS8C6zdxzw9Ju8kAE0hdBcvpJek3wMpAF+DuwrGbbS8Ansol6JpF0lDSPwjo0LV7cy8LIYQQQiPmvfVSrZdQ1yJgbkWNNDmZwCebnKjMWGm8nHl8/P8hKO46FxulfKLpScEIYNdcbm4IqVRdSbFxSmONTD4mGpeEEEII1bfazkfXegl1LQLm1tVYk5ODJY2xPU/SKsDTwJqS+tl+XNJKpJSMB0kNUe6XtD6wNiknuitwqKRlgLWArZqxnlJzkpKVgFckLZvv0dQ/Xxte36hoXBJCCCGE9iBymFvXYOCmBmM3AGsC/yY1GJkMfNf2h8DewHl57F7SrvGFQAdJU0k5zkNsf0BqOvICMBU4gxSEN2UUqdnJE5LWBX5N6up3Lylgb8oUYJ6kyZKOasb5IYQQQqiC448/nuOPP77Wy6hbUSUjtJpoXBJCCCFUxwEHHADA5ZdfXuOVtF+NVcmIlIzwEUnzSTvWHUm719+z/Y6knvnzEbbPy+eeD4y3PaI2qw0hhBDqRwTKtRUBcyiaY7s3gKQrSCXtTsnHXgd+IumPOX2kSVNfmknPYXe0zkpDCCGE0GxRf3nxRA5zqOTvpJcJS94A7gP2r81yQgghhPr19gMjePuBEbVeRt2KgDl8Qu5E+FVSo5Oi04Cf5eOVrh0qabyk8fNnz2zNZYYQQgh1Y8Gc91gw571aL6NuRUpGKOokaRLQk1Qr+t7iQdsvSHoM+G6lCaIOcwghhFB9q+54eK2XUNdihzkUlXKYPwcsR27L3cBvgV8Q/+2EEEIIoU7EDnP4BNszJR0B3CLpogbHnpb0FLAT8Fhj80TjkhBCCKE6jj46dfo744wzaryS+hS7hKEs208Ak4F9yhw+BfjMkl1RCCGEUL/mzJnDnDlzar2MuhU7zEuRBnWQ/wHsb3v2Ys7ZE7jddq+mzrXdpcHnnQsfexXGJ0u6Epi1OGsLIYQQQvNccMEFtV5CXYsd5qXLHNu9c3D7IXBwcy6SFP/wCSGEEEJoJRFoLb0eAjaTtDPwK9JLeG8C+9p+TdJwYE1SRYsZko4FrgJWzNcfZvuR4oS5HNxpwABgeeAC23+UtAxwPvBlUke/ZYDLbF8v6XhgZ6AT8AjwIzezn3o0LgkhhBCq462/XQLAKoOGtvjaaFqy+GKHeSmUd4y/QUrPGAt8yfYWwCjg54VT+wC72P4uqRPf12xvCewNnFtm6gOBmbb7Af2AgyR9HvgOKfDeFPgh0L9wzfm2++Vd706kl/1CCCGEEOpG7DAvXUp1kCHtMP8Z2AC4VlIP0i7zC4Xzb7VdegNgWeB8Sb2B+cD6ZebfgbRrvUf+3A1YD9gWuM72AuBVSaML1wyU9HOgM7AKMA24rdIDSBoKDAXo0LV78546hBBCCI1alJ3lUD0RMC9dSnWQPyLpPOAs27dKGgAMLxz+X+Hno4DXgM1J/8/B+2XmF3C47bsb3KPs/1cjaQXgQqCv7f/kNJAVGnuAaFwSQgghhPYmAualXzfgpfzz/k2c91/bCyTtD5RrX303cIik+23PlbR+nnsssL+kK4DupBznq1kYHM+Q1AXYA7i+uQuPOswhhBBCdfz4x6mXWFTLqI0ImJd+w4HrJL0EPAp8vsJ5FwI3SNoTGM3Hd59L/kTKVZ4oScAbwK7ADcBXgSeBZ4FxpFzndyRdSsqlng48Xp1HCiGEEEJLdOrUqdZLqGtqZsGD0M5J6mJ7lqRVSR38trH96uLM2bdvX48fP746CwwhhBBCaEWSJtjuW+5YVMlohKTjJE2TNEXSJElfbOL8EYUX6hb1ngMkWdKBhbEt8tjRizN3E27PLxw+BJy8uMFyCCGEEEJ7ESkZFUjqTyqhtqXtDyStRqpSsSRMJZWG+3P+vA+pTXWrsT2gpddI6mh7XissJ4QQQggFQ4emKhmXXHJJjVdSnyJgrqwHMMP2BwC2Z5QONKeZh6Q+wFlAF2AGMMT2K5KOIHXwmwc8ZXufMvf+N9BV0uqk+so7AncW5j6IVLptOeA54Hu2Z0saAbwL9AXWAH6em490AW4BPkUqP/cr27fkuX4N7Av8J69zgu0zJK0LXEB6CXA2cJDtp/M93gK2ACYCP6v0BUbjkhBCCKE63p76NgD3tPDv1WhaUh2RklHZPcBnJT0r6UJJXy4ca7SZh6RlgfOAPWz3AS4DTsmHhwFb2N6MxltfXw/sCWxNCkw/KBy7Md9/c+AfpIYkJT1IdZV3InX1g1Ribrfc1GQgcKaSvsDupOD3O6RAu+QSUgm6PsDRpJcKS9YHBtmuGCyHEEIIoXo+9eUhfOrLQ2q9jLoVO8wV5Bfg+gDbkYLMayUNsz2Cppt5bAD0Au5NxSjoALySj00BRkq6Gbi5kSX8BbgW2BC4hhQ4l/SS9BtgZdIOdrGu8s25AclTeYcaUv3l30raHlgArAWsTgqsbyk1P5F0W/69S77fdXn9kFppl1xne365RUfjkhBCCCG0NxEwNyIHhWOAMZKmkmoVj6LpZh4Cptnuzyd9C9ge+Dbwa0mblMsDtv2qpLnA14Cf8PGAeQSwq+3JkoaQ6iaXFHeiS9HuvqTUij65/vL0vGZR3jLAOw2bqBSUK1lXWnc0LgkhhBCqbMYdZwOw2reOrPFK6lMEzBVI2gBYYPufeag38CLNa+bxDNBdUn/bf88pGuuT0ic+a3u0pLHAd0k7xO9UWMbxwKdtzy/s9AKsBLyS592XhY1NKukGvJ6D5YHA5/L4WOCPkk4l/bfwLeBS2+9KekHSnravyzWbN7PdohcPo3FJCCGEUB3HLzcOgJNOir9XayEC5sq6AOdJWpn0gt5zwNDmNPOw/WEuL3eupG6k7/lsUlOQ/8tjAv5gu1KwjO1HKhz6Nam5yIt5HSs18SwjgdskjQcmAU/n+R+XdCupAseLwHhgZr5mX+AiSb8ivSg4ilau1BFCCCGE8k466aRaL6GuReOSOldoWNIZeJD0j4KJ1Zg7GpeEEEIIoa2IxiWZpN1yA5ANl4K17Cpp4wrHhud1fqEwdlQeK/sHuRguyQ1LJgI3VCtYDiGEEEL17Lfffuy33361XkbdqreUjMGkvN19gOG1XQq7ArcDT1U4PpW0zt/kz//P3p2H2zWe/x9/fxpBIkSLbxuqDa2hRISENhTRqm9VTUUNVUJ/xpZqRZvWFFFTac204UuMTSoxT0GIKdSUSESFIq2iiCGkIiI+vz+eZ8vKzt5nSPY5Ozn7fl1XrrP3Ws961rN3uNznca/73q2JsQvN9t6Lcr2kTtUqZkQd5hBCCKE23v1n+tnS/65G/eXaapgd5vyA3uakmsV7Fo53knSmpMm5Bfbh+fgmksZLekrSo5KWl7SspMvy2An5ATokDZR0fmHOWyQNyK9nSjo5z/OIpM9L2oxUJeOM3HL7KxWWfAOwU55jTVJu8ZuFe1wk6fHcuvvEwvFpkk6U9GRe57r5+Kb580zIP9fJx7tK+mv+7CMl/a20iy1pW0kP57muzd9h6R7H5wcXd1/Ev5oQQgghNGPFzfdixc33qvcyGlbDBMykHd07bD8HvC1p43z8IGAN5jUTuVrS0qQayD/PzUG2AWYBPwWwvQFpt/pySeUl5cotBzyS57mf1DFvPHATcLTtPrZfqHDde8DLknrle40sO39MzrPpDWwlqXfh3PTcpOQiUtMRSA/6bWl7I1L1jVPy8cOAd/JnPwnoC6DUCvxYUoOSjUkPBP6ycI8PbX/T9ohmPn8IIYQQwhKtkQLmvUiVHsg/S7+mbQP8qVQL2fbbpMYjr9l+LB97L5//JnBlPvYsqbLE2s3c9yNS6gXAE0DPVqx5BGk3fGfg+rJzP5T0JDABWB8o5kNfV+F+3UmNSJ4GzsrXQPpMIwBsP01qrALwjTznQznHeT/mlaODBQN4IDUuyTvfj8/9YEalISGEEEJopTdvPJ03bzy93stoWA2RwyxpJeBbpA55JnXec+7WJ6C8VEilY6XjlXzM/L98FHed53heKZK5tO47vxk4A3g810ZOi5DWIO0cb2L7HUnDy+5Zal5SvN9JwL22d5HUk9SQpanPJOAu29X+/0/F5iXRuCSEEEKovaU/v2a9l9DQGiJgJj0wd4Xtg0sHJN1H2l29EzhE0jjbH0v6HCl9YVVJm+RaxcuTUjLuJ9UnvkfS2sCXSE1KVgAOk/QZUtvpTVuwpvdppn6y7VmSfk2q31y0AilgnaHU/no75gXA1XRnXoOTgYXjDwI/BO7NVTs2yMcfAS6Q9FXb/8hl576YU1paJBqXhBBCCLUS/z2tp0ZJydiLBVMaRpM67V0C/AuYJOkpYG/bHwF7kBqXPAXcRdrBvRDopNQmeyQw0PZs4CHgJVJlizNJJdqaMwI4Oj+EV+mhPwBsjygv9ZY77k0ApgCX5vs35/fAqZIeIu2wl1xI6ko4Cfg1KSVjhu03SYH1X/K5R4C6yRPXAgAAIABJREFUl+MLIYQQQmhv0bikwUnqBHS2/WEO3McCa+dfGhZJNC4JIYQQamPXXXcFYPTo0XVeScfVVOOSRknJWKLlHOyx+e0XSLnJpRJzmy5scJtzmW8DPpDUmZS3fGgtguUQQggh1E7//v3rvYSGFjvMSxhJQ4CZts+swVw9gVts91rI65cqVRepZJkea7nHfmcv5OpCCCGE0FrRsGThRWvsDkhSX0n3SXpC0hhJPfLxAyU9lhuljM4P65Ebplyfjz+Vm6dAysm+ODdAuVNSlzz+K5LuyPM/UGiAMlzSHyXdC0R9mxBCCCF0eBEwL5kEnAfsZrsv6cG/k/O562xvkhul/J3U2RDgXOC+fHxj0gODAGsBF9heH3gX2DUfHwYcnucfRHo4sGRtUkOToxZYWNRhDiGEEGrujdFDeWP00Hovo2FFDvOSaRmgF3BXrs3cCXgtn+sl6XfAikA3YEw+/i1gXwDbc0kl6T4LvGR7Yh7zBNAzt8DejNTopHjPkmvzHAuIOswhhBBC7S375Q3rvYSGFgHzkknAFNuVngAYDuxs+ylJA4EBzcw1u/B6LtCF9H8e3rXdp8o1FZuWlIs6zCGEEEKtxH9P6ylSMpZMs0m1k/sDSOosqdTqenngtVz14keFa8YCh+bxnSStUG1y2+8BL0naPY+XpPjVNoQQQggNKQLmJdMnpO6Fp+fGKhNJKRQAxwF/IzVbebZwzc+BrXPTlSeA9Wnaj4Cf5PmnADvVbvkhhBBCaI3tttuO7bbbrt7LaFiRklFG0heBC4D1SL9Q3AIcXapNLOkvpGDzMuB2Usc+kwLYK21vVmneVtx/YJ57G9tj87FdgOuA3W0PKQzfsvx62xcBF1U4/jqVg95ehTFnFl6/BHy3wjwDW/ZJQgghhFArO+ywQ72X0NAiYC5QesLtOuAi2zvlLnjDSBUojpb0BWAz21/O4wcDN9o+IU+xSMFywWRSO+9Ss5I9gadqNHfNNFeHOYQQQgi1cdhhh9V7CQ0tAub5fQv40PZlkKpJSPoFKZ/3BOBO4H8kTQSuJ+UEz5W0pe2tJc203Q1A0q+AH5PSJ263PTi3nr4AWAX4ADjQ9rPliwAeALbIecjLAF8lpV2Q5z4e2IH0gN544GDbljSOlI6xNalKxk9sP5AblFwJLJen+Jnt8ZI+A5wPbAW8RNpRv9T2KEl9gT+SKm1MBwbafi3fYzywOXAT8IdqX+bkV2bQc/CtzXzlIYQQQqiVaFzSNiJgnt/6pPzeT9l+T9K/SEHrjqTOeH3g0x3pBbruSdoO2Bn4uu0PJH0unxoGHGL7eUlfJ9U2/laFdRi4G/hfoDspMF2jcP5820Pzva4Evg/cnM8tZXtTSd8DTgC2Ad4AvmP7Q0lrAX8B+gE/AHoCGwD/Q6rbfGkO1M8DdrL9pqQ9SLvsB+R7rGh7q2a+yxBCCCHUyOsjjgHg83ue3MzI0BYiYJ6fSMFqS49Xsw1wme0PAGy/3YLaxuVGAEeQAuajgN8Wzm2dd7C7Ap8jPZRXCpivyz+fIAXDAJ2B8yX1IZWOWzsf/yappvInwH9y9z6Adahe5xlgZLVFSzoIOAig0wqrNPHxQgghhNBSXdfdot5LaGgRMM9vCvM63QGQy6+tDrxA2oVtiUoBdnO1jedj+1FJvYBZtp8rBdmSliXtTPez/bKkIcCyhUtLdZXnMu/v9xfA68CGeR0fFtZZbf3V6jxDE3WYo3FJCCGEUHvL91ngOfzQjiJgnt9Y4DRJ+9q+Ij/09wdgeE6taOk8dwLHS7qmlJKRd5lfkrS77WtzOkdv2009zPcb5gW3JaXgeHretd4NGNXMeroD/7b9iaT9SDvGAA8C+0m6nJRXPQC4BphKrvNs++GcorG27SkV5q4qGpeEEEIIoSOIOswFtg3sAuwu6XngOVLA+tsmL1xwnjtIeceP5wcEB+VTraptbPt22/eWHXsXuJhUSeMG4LEWLOlCUmD8CCkdo7RDPBr4N/A08GfSA4Mzcgm9anWeQwghhNDOBgwYwIABA+q9jIalFCOGRiWpm+2ZklYCHgU2t/2fWszdr18/P/7447WYKoQQQmhow4cPB2DgwIF1XUdHJukJ2/0qnWvzHWZJu0iypHWbGDNOUsUFVhnfT9K5LRg3vpnzrdo5Llx3pKSuhfe3SVpxYeaqMPdG+fv634W8/hBJ+zZxfoCk4m7xLXkX/AHgpOaCZUkrSopikCGEEEI7GjhwYATLddTmO8yS/gr0AMaWdakrjhkHDLLdrtuRxbrJZcdF+m4+qXLdNNJDd9PbYE2/B/oDL7RFV738kOACpfBacX1PUmm9Xs0MZZkea7nHfmcvzG1CCCGEUOC5qU+YOlV//CxqMC+auu0w54fSNgd+QupWVzreRdIISZMkjSQ14CidmynpdElPSLpb0qZ5B/pFSTvmMQMk3ZJfD5F0aWHMEcW58s8eku6XNFHS05K2kHQa0CUfu1pST0l/l3Qh8CSwuqSLJD0uaYqkE/NcRwCrAveWyrBJmiZp5bzuwwr3HyLpqPz6aEmP5c98YpXvS6Tc4YHAtrkiBpKWk3SrpKfy+vfIx0+T9Eye88zCPQeV1lo4PyIHu4cAv8ifewtJO0j6m6QJ+fv+fDPf62nAV/L1Z7TiH4cQQgghLKTXRx7L6yOPrfcyGlZbV8nYGbgjl0V7W9LGtp8kdcj7wHZvSb1JAWrJcsA427+WdD3wO+A7wHrA5aSH6cqtS+putzwwVdJFtucUzu8NjLF9slLli665A97PCk1IepLqD+9v+7B87Jhc3aITMFZSb9vnSvolsHWFHeYRwNmkh+wAfgh8V9K2wFrApqSSbTcpdQe8v+z6zYGXbL+Qd92/R6qr/F3gVdvb53V1V2qGsguwbu7yVyklZDCwhu3Zkla0/a6kP1HYYZb0WeAbeY7/B/yKVPe54vea5+zV0vJ4IYQQQlh03TZcqEzNUCOt3mGWtFzzoz61FymIJP/cK7/eErgKwPYkYFLhmo+AO/LrycB9OfidzLxGHOVutT07B7BvAJ8vO/8YsH9OR9jA9vtV5vmn7UcK738o6UlgAqkL4HpVriN/lgmk1tmrStoQeMf2v4Bt858JpF8O1iUF0OWqfV+TgW3yDvYWtmcA75EqeFwi6QekVtvlJgFXS9oH+LjKsr8IjJE0GTg6f86S5r7XBUg6KO/KPz73gxnNDQ8hhBBCC3Rbf2u6rb91vZfRsFocMEvaTNIzpPbJSNowpy9UG78Sqe3zJTnn92hgj5x2ANU7583xvMTqT8iNOHI+cbUd8dmF13PLx+Wd3C2BV4ArVf2huE8bckhag1QO7tu2ewO3Mn+DkGpGkdIq9mBe8CvgVNt98p+v2v6/4kV5F3tXUv3maaTW1NtJWt72c0BfUuB8qqTjbX9M2rEeTd7Jr7CW7YEL8rVPSKr0/Z1HarW9AXAwlZugQIXvtRLbw2z3s92vU9fuzQ0PIYQQQgt8MudDPplT3pohtJfWpGScBfwvOSXC9lOStmxi/G7AFbYPLh2QdB+pHfP9pJrE9yp1s+vd2oW3hqQvA6/YvjjvkG8MXAHMkdS5LH2jZAVSAD0j5/VuB4zL594npSlUeuhvBKlO8srAVvnYGOAkSVfnEm6rkX4xeKNw3TbAU7Y//X8uSg1FdpY0Fnjb9lU5L3ugUn54V9u3KdVX/kfZZ/4MsLrteyU9SEpL6ZbXvkJhaHfSLxIA+1X4POVKn71Z0bgkhBBCqI1SDeZx48bVdR2NqlU5zLkVc/HQ3CaG70V6QKxoNClw+yVwmaRJpKYYj7ZmHQthAHC0pDnATKC0wzwMmJTTLo4pXpB/IZhAajDyIvBQ4fQw4HZJr9neuuy6KZKWJwXor+Vjd0r6GvBw/v5mAvuQ0hxK9gKuL1v3aFK+9xvAGZI+AebkY8sDN+YHA0Vqf13UCbhKUvd8/qycw3wzMErSTsDhwBDgWkmvAI8Aa1T5Dkuf7y1JD0l6Grjd9tFNjQ8hhBDCojv00EPrvYSG1uKycpJGAX8Ezge+ARxBKq22Z5MXhoYVjUtCCCGEsKRQjcrKHQL8FFiN1E65T36/xJA0V/NKy91cpbJES+ZZNf8CUe18mzb3KJaOq3D8A0n/Uzg2s63WEUIIIYT2MWPGDGbMiIfp66XFKRm5UsKP2nAt7WFWoYzc5aSA/+TWTmL7VVKOdjUrAocxr7xcs/LDkFWbpbTCdFJZuF8vzMWSlsoPFC6yya/MoOfgW2sxVQghhNDQ/nPNYAC+sPe8bNdoVNJ+mg2YJZ1H9YoW2D6i2rnF3MPkhw0lfYVUTWIVUnm2A20/m49fTcoHvh34pe1uKnS7k7Q+cBmwNGnHflfgJHJzD+Au20dLOppUl3kZ4HrbJ+R5bgfuJXX321nSD8vH5TUeQ8q9fhl4E3iiyue6lPRQ4Om23y4dVFmHvrxD3c32kFzzeTypDvRNkv4FnEDKUZ9he8tcxeM0Uj74MsAFtv/cqm88hBBCCAtlhb471nsJDa0lO8wdLgk1B3/fBkql3YYBh9h+XtLXSTvD3wLOAc6x/RdJh1SZ7pA85mpJS5OC6/mae1RrXAL8i0KzlCbG/ZfUKXEj0t/Zk1QPmGeSguafk4LellrR9lZ5vZOB/7X9SiFt5Sek4HkTScsAD0m60/ZLxUkkHQQcBNBphVVacfsQQgghVNN1nc3qvYSG1pK6upcDSNrd9rXFc5J2b6uFtZEuede3JyngvCuXZ9uMVCmiNG6Z/LM/qcYxwDXAmRXmfBg4RtIXgety0F0+pti4BFJ5t7VIAXOxWUq1ccuTdps/AJBUqdth0bnAREl/aGZc0cjC64eA4ZL+Suo0WFpbb0mlVJTueW3zBcy2h5F+AWGZHmu17InSEEIIITSp1AwsehzUR2vKyv0GuLYFxxZns2z3yaXWbiHlMA8H3l3YVs+2r5H0N1KTkDG5vfSLZcNKjUvmS2HIaRL/bcG4I2kiLabCmt6VdA0pj7rkY+Z/yLO8Ccun67B9SN5p354UePfJazvc9piWriPqMIcQQgi1EXWY66vZKhmStst5zKtJOrfwZzjV2y0v1nJr6SNInfxmAS+VdsuVbJiHPkLKSYaUErEASWsCL9o+l9TUpTcLNvcYAxyQd7ORtFqxkkULxt0P7CKpS67xvEMLPuYfSZ37Sr8UvU5q271STqn4frULJX3F9t9sH096iHD1vLZDJXXOY9ZW69qkhxBCCGEhHXXUURx11FH1XkbDaskO86ukPOYdmT9v9n0WbJaxxLA9QdJTpED4R8BFko4FOpO69T0FHElq/nEUqTV2pXouewD75KYo/wGG2n67vLlHlcYl8zV+qdbgxPaTkkaSmrz8E3igBZ9vuqTryX9HtudIGgr8jZRG8WwTl58haS3SrvLY/F1MIqWyPJkrerzJvHSVEEIIIbShHXZoyV5ZaCutaVxSs3JjSwpJXUlpHJa0J7CX7Z3qtJYVgb1tt7hUXQvmHALMtF0pN3uRReOSEEIIoTb+85//APCFL3yhzivpuJpqXNKSsnJ/tf1DYIKkBaJr271rsMbFVV/g/Lyj+i5wQB3X0urazm1NUifbTbVHDyGEEEIN7LlnygyNHOb6aElKxs/zz6o5rx2V7QeADZsd2D5Oo1DbGXiDyvWabyDlHC9LKnc3LB//LnAKqezddNvfzvOul+swfwk4O+diI2kfUp730qQ0jsNsz82dA/8I/C+pQcqD1RYcjUtCCCGE2vjT4MH1XkJDa0lZudfyy8Nsz9c9TtLpLGRHudBqn9Z2zvWad6OsXrPt+4EDcg51F+AxSaNJD3deDGxp+yVJnyvMuy6wNekhxamSLgK+SsrN3jznPl9IyvO+AlgOeDo/EBhCCCGEdvDd73633ktoaM1WySj4ToVj29VqIaFVivWanyQFvWvlc0fkhxkfIe00rwV8A7i/1GSk2AEQuNX27Nz6/A3g86SmLn1JAffE/H7NPH4uMLrawiQdJOlxSY+XakaGEEIIYdG8/PLLvPzyy/VeRsNqSQ7zoaTc2TUlTSqcWp7U4CK0v2r1mgcA2wD9bX+QUy2WzeOrPd05u/B6LumfCQGX2/5NhfEfNpW3HI1LQgghhNr78Y9/DEQOc720JIf5GuB24FRSWkDJ+8WdSkmftf1OjdcX5inWdh4DnCTpatszJa0GzCF133snB8vrknaWIXUjvEDSGqWUjLJd5nJjgRslnWX7jZzCsbztf7ZmwdG4JIQQQqiNu+9epvlBoc20JId5Bqn+8F7NDB0LbFyLRYUF2X6rWNuZ9ItMeV3nO4BD8v8JmEpKy8D2m5IOAq6T9BlS6kWlFJvSvZ7JNanvzOPnkLoitipgDiGEEEJtbLPNNvVeQkNrcR3mZieSJtjeqCaThQ4h6jCHEEIItfHiiy8CsOaaazYzMiyspuowt+ahv+Ys1vmqkj4v6RpJL0p6QtLDknap43oGSjq/ynFL+nbh2C752G7tu8oQQgghLA4OOOAADjignu0gGltLcpiXeLnxyA2kB9n2zse+TGr33Zb3XdjuiJNJKTBj8/s9Se2pFyuN2P0xhBBCqIcTTzyx3ktoaLUMmFXDuWrtW8BHtv9UOpAfYDsPUsc6UmOQAaRGIBfY/nOuOjEEmA70Ap4A9smtsvuSGnh0y+cH2n4tV6YYD2xOqo/8HHAsqQHIW8CPbL/ezHofALaQ1Dmv56vAxNJJSccDOwBd8r0OzmsaR2oysjWpM+BPbD8gqSdwJamGMsDPbI/P+cnnA1sBL5H+j8Oltke19PMBf6j2IaJxSQghhFAb0+Ih+rpqVcAsaWPgm6T0i4dsP1k4/e3KVy0W1ifVK67mJ8AM25tIWgZ4SNKd+dxG+fpXSWX0Npf0N1KwvVN+oG4P4GTmtc5e0fZWkKqHAN/IAe3/A35F6pDXFAN3k7rpdScFpmsUzp9ve2ie/0pSF8ab87mlbG8q6XvACaQyc28A37H9oaS1gL8A/YAfAD2BDYD/Af4OXJoD9RZ9vhBCCCG0valTpwKwzjrr1HkljanFAXPe1dwduC4fukzStbZ/Bws0w1isSbqAFPh/ZHsTUhOQ3oUc4e6khh8fAY/a/ne+biIpwHyXtON8V65S0Ql4rXCLkYXXXwRGSupB2mV+qYXLHEFqTd2dFGD/tnBua0m/AroCnwOmMC9gLv39PJHXCtAZOF9SH1Kt5bXz8W8C19r+BPiPpHvz8XVa8fnmk6txHATQaYVVWvhRQwghhNCUgw8+GIg6zPXSmh3mvYCNbH8IIOk00q7t79piYTU2Bdi19Mb2TyWtDJRKOAg43PaY4kU5JaNaY48ptvtXud9/C6/PA/5o+6ZCikezbD8qqRcwy/ZzOXBF0rLAhUA/2y9LGkJqTlJSWm9prQC/AF4HNiSlXXxY+ohVbt+az1e+7mhcEkIIIdTYKaecUu8lNLTWBMzTSIFZKdhaBnih1gtqI/cAp0g61PZF+VjXwvkxwKGS7rE9R9LawCtNzDcVWEVSf9sP5xSGtW1PqTC2e2Gu/Vq57t8w7/suKQXH0yV1A3YDRjUzT3fg37Y/kbQfaccY4EFgP0mXA6uQcrivoXWfr6poXBJCCCGEjqA1AfNsYIqku0g5tt8BHpR0LoDtI9pgfTWR84d3Bs7KqQxvknZJf52HXEJKX3gyV9R4E9i5ifk+yukb50rqTvoezybtZJcbAlwr6RVSI5E1Koypdp/bKxx7V9LFpEoa04DHWjDVhcBoSbsD9zJvh3g0Kff8aeA50gODM1r5+UIIIYTQxp5++mkAevXqVeeVNKYWNy7JO5NV2b68JisK7UpSt9xeeyXgUWBz2/+pxdzRuCSEEEKojQEDBgCRw9yWmmpc0uId5o4SEEuaS9qdXYr0AN6Pbb+7EPOsCpxru2IzEUkrAnvbvnBR1tvE/YcAM22fuYhT3ZLXujRwUq2C5RBCCCHUzhlnnFHvJTS01lTJ+D5wEvBl5j34ZtsrtNHa2sos230Acu7uT0kl01rF9quk/OFqVgQOI6VDtEhOB1GuWtEubA8oW0Mn23NrMXfUYQ4hhBBqI+ow11drWmOfTXpobSXbK9hefgkMlss9DKwGIOkrku7IbbMfkLRu4fgjkh6TNFTSzHy8p6Sn8+v1JT0qaaKkSbnW8WnAV/KxM/K4o/M8kySdWJjn75IuJFUdWb3SuDz2GElTJd1NKv22AEnDJZ0rabxSG/Dd8vEBkm4pjDtf0sD8epqk4yU9COwu6QhJz+T7j8hjlpN0aV7XBEk71e6vIYQQQghNmThxIhMnTmx+YGgTrXno72Xgabc06Xkxp9Td79vA/+VDw4BDbD8v6eukneFvAecA59j+i6RDqkx3SB5ztaSlSVUoBgO9CrvZ25JqO29K2p2/SdKWwL9Iwe/+tg9rYtx/SS2yNyL9vT1JqrVcSQ9SjeV1SU1PmquiAfCh7W/mtb4KrGF7dk7XADgGuMf2AfnYo5Lutj1fibmowxxCCCHU3pFHHglEDnO9tCZg/hVwm6T7KNQmtv3Hmq+qbXUpNCB5gtScoxuwGamaRWncMvlnf+ZVzLgGqJQz/DBwjKQvAtfloLt8zLb5z4T8vhspMP4X8E/bjzQzbnngetsfAEi6qYnPeENO63hG0uebGFdUbEYyCbha0g3ADYV17ShpUH6/LPAlUnfAT0Ud5hBCCKH2zj777HovoaG1JmA+GZhJCpSWbpvltItZtvvkcmm3kHKYhwPvlnaDW8v2NUrtsrcHxii1wH6xbJiAU23/eb6DUk/mbwRSbdyRpHJ+LVFstlKK3D9m/hScYrMTytawPbAlsCNwnKT18zy72p7awjWEEEIIoUb69FmoECXUSGsC5s/Z3rbNVtLObM+QdARwI3AR8JKk3W1fmx++6237KVLt5F1JO7B7VppL0prAi7bPza97A0+RdoVLxgAnSbo6l3FbDZhTYbpq4+4Hhit1WFwK2AH4c4Xrq/knsJ6kZUjB8rdJjUvKP8tngNVt35tzmvcm7XKPAQ6XdHiua72R7Qnl1xdF45IQQgihNh57LLVd2GSTTeq8ksbUmoD5bknb2r6zzVbTzmxPkPQUKRD+EXCRpGOBzsAIUtB7JHCVpKOAW4EZFabaA9hH0hzgP8BQ229Leig/GHi77aMlfQ14OKdrzAT2IbWwLq7pzkrjbD8paSQwkRT8PtDKz/qypL+S0i2eZ17KR7lO+fN2J+0qn5WbpZxEevBzUv6FYhrw/dasIYQQQggL5+ijjwYih7leWtO45H1SO+mPSDueS2pZuVaR1JWUxmFJewJ72a55hQjNXx/678B+pXzlRZjzNlIt6HfLjg9hIWs45xSSzWxf09zYaFwSQggh1EZ0+mt7qkXjEqA7aRd2DdtDJX2JVI2ho+sLnJ93Vd8FDmij+xTrQ19Nqrzx6QOVWoj6yLa/V9slAulhyb1JD0CGEEIIoR1EoFxfrQmYLwA+IZVaGwq8D4wGOnQyje0HgA3b+bYPAL0lDQBOAF4D+kjagFTfeQCpiscFtv8sqQcpx3oF0t/pobYfkDQN6Gd7uqRjgH1J5QHfJJekk/QV0t/tKsAHwIG2n5U0HHgP6Ad8AfiV7VH5/l/LlUYut31WtQ8RjUtCCCGERTfttO0ZP348AJtttlmdV9OYWhMwf932xpImANh+J9ccDjUkaSlgO+COfGhTUj3nl3KN4xm2N8kP7z0k6U7gB8AY2yfn+tJdy+bsS/UaztXqT0Ples6DgUG2I385hBBCaCe//e1vgchhrpfWBMxzcjBmAEmrkHacQ22U6kND2mH+P1Jt6Edtv5SPb0vaeS615O5OqtH8GHCppM6kGszlrYC2oEIN52bqT8NC1HOOxiUhhBBC7f35z60pjBVqrTUB87nA9cD/SDoZ2A04tk1W1Zg+zWEuyUFseY3mw22PKb84dwPcHrhS0hm2rygbUunpzs/QdP3pSvWcmxSNS0IIIYTaW2eddeq9hIbW4oA5t31+glS/V8DOtv/ezGWhtsYAh0q6x/YcSWsDrwArA6/YvljScsDGQDFgrljD2fZ7kqrVn67mfeavL11V1GEOIYQQauO+++4DYKuttqrzShpTa3aYsf0s8GwbrSU07xJSlYonc3D7Jqlt9wDg6FwHeibp4b5PNVPDuVr96WomAR/n+tXDm3roL4QQQgi1ccIJJwCRw1wvLa7DHEJrRR3mEEIIoTZefPFFANZcc806r6TjaqoO82faezGheZIs6crC+6UkvSnplnquK4QQQgj1seaaa0awXEcRMC+e/gv0ktQlv/8OKVd5sZJL4IUQQgihjd19993cfffd9V5Gw4qUjMWQpJmkqiRP2h4l6QpgCrCF7e9L2hQ4G+gCzAL2tz1V0kBgR1Id5q+QSsn9Ks95EanJTBdglO0T8vHvkToKTifVZ14z32M54DxgA1Ku+xDbN+Z7bA8sCyxnu1SzeQHL9FjLPfY7u5ZfTQghhNBQpuWH5wcMGABEDnNbqlVr7NC+RgDH5zSM3sClpHrKkB683NL2x5K2AU4Bds3n+pAalMwGpko6z/bLwDG23861tMdK6g08B/w5z/WSpL8U7n8McI/tAyStCDwqqfSrbX9SNY232+rDhxBCCGGeK6+8svlBoc1EwLyYsj1JUk9gL+C2stPdgcslrUWqr9y5cG6s7RkAkp4Bvkxqh/3D3FRkKVIHv/VIKTkvFhqj/IXcdITUJGVHSYPy+2WBL+XXd1ULlqNxSQghhFB7q6++er2X0NAiYF683QScSSobt1Lh+EnAvbZ3yUH1uMK5YrORucBSktYABgGb5Jbmw0kBcFPNSATsanvqfAdT++z/Vr4kGpeEEEIIbeGOO+4A4Lvf/W6dV9KYImBevF0KzLA9WdKAwvHuzHsIcGAL5lmBFOTOyC2utyMF2c8Ca0rqaXsasEfhmjHA4ZIOt21JG9me0JrFR+OSEEIIoTZOO+1r6WcZAAAgAElEQVQ0IALmeomAeTFm+9/AORVO/Z6UkvFL4J4WzPOUpAmkBwdfBB7Kx2dJOgy4Q9J04NHCZSeRHiyclJukTAO+vwgfJ4QQQggLacSIEfVeQkOLKhkNTlI32zNzUHwB8HytuvdF45IQQgghLCmiccliRNIxkqZImiRpYs4Jrsc6xueXB0qaSNp97k6qmhFCCCGExcjNN9/MzTffXO9lNKzYYW5HkvqTah4PsD1b0srA0rZfbYN7ifT3+0mt5y7co5PtudXORx3mEEIIYdFEHeb2EzvMi48ewHTbswFsT7f9qqRpOXhGUj9J4/LrIZKulHSPpOclHViaSNLRkh7LO9Un5mM9Jf1d0oWkJiTHSfp94ZqBks7Lr2fmnz0k3Z93u5+WtEU+vpekyfnY6YU5ZkoaKulvpHrMIYQQQmhjo0aNYtSoUfVeRsOKgLl93QmsLuk5SRdK2qoF1/QmddbrT2pksqqkbYG1gE1JjUr6Stoyj18HuML2RsCFwA8Kc+0BjCybf29gjO0+wIbAREmrAqcD38rzbyJp5zx+OeBp21+3/WD5YiUdJOlxSY/P/WBGCz5eCCGEEJqz8sors/LKK9d7GQ0rAuZ2ZHsm0JfU2ONNYGRuNd2UG23Psj0duJcUJG+b/0wg7SSvSwqgAf5p+5F8vzeBFyV9Q9JKpGD6obL5HwP2lzQE2MD2+6QW2uNsv2n7Y+BqoBSQzwVGN/EZh9nuZ7tfp67dm/loIYQQQmiJ6667juuuu67ey2hYUVauneWc33HAOEmTgf2Aj5n3y8uy5ZdUeC/gVNvzPaCXm5iUNxUZCfyQVHP5epclrdu+P+9Obw9cKekM4L0mPsKHTeUthxBCCKH2zj33XAB+8IMfNDMytIUImNuRpHWAT2w/nw/1Af4JdCHtPN8O7Fp22U6STiWlQgwABgOzgJMkXZ1Lwq0GzKly2+uAY/J9fl1hTV8GXrF9saTlgI1J6Rjn5Lzqd0jtuc9r7eeNxiUhhBBCbdx44431XkJDi4C5fXUDzpO0ImlX+R+k9IyvAf8n6bfA38queRS4FfgScFKuqPGqpK8BD6diGMwE9iGlS8wnt8J+BljP9qPl50lB+NGS5uR59rX9mqTfkFJABNxmO/5NDSGEEOqke/dIc6ynKCvXznIu8dj89gukIPfN/H5T2x8Vxg4BZto+s3BsADDIdou77knakRQwn9bEmPG2N2vpnC0RjUtCCCGE2hg5Mj2zv8cee9R5JR1XU2XlYoe5ndl+i5SKUTEgbqN73gTc1MyYmgbLIYQQQqidiy66CIiAuV4iYF4MSOpLamjSDZgODLT9GnAV8CdJPybtRO+eL+kmaRTQC3gC2Me2JU0DLgd2ADoDu9t+Nlfi6Gf7Z5I+D/wJWDPPdajt8ZJm2u4mqRtwI/DZPMextm/MDxTeDjwIbAa8Auxke1a1zzX5lRn0HHxrDb6hEEIIoTGVGpfcdtttdV5JY4uycvUn0gN1u9nuC1wKnJzPXQ1cYHtDUpD6Wj6+EXAksB4p8N28MN902xsDFwGDKtzvXOC+POfGpJbYRR8Cu+Q5tgb+kLsGQipdd4Ht9YF3WfABxRBCCCG0ga5du9K1a9d6L6NhxQ5z/S1D2im+K8elnYDXJC0PrGb7egDbHwLkMY/a/nd+PxHoSdr5hVQVA9LOc6XaM98C9s1zzgXKu4sIOCWXmvsEWA34fD73ku2Jhfl7lk8u6SDSg4x0WmGVFnz8EEIIITTnqquuAmCfffap80oaUwTM9Sdgiu352kxLWqGJa2YXXs9l/r/H2VWOt9SPgFWAvrbn5DSPUm3o8vt2Kb/Y9jBgGMAyPdaKJ0pDCCGEGrjkkkuACJjrJQLm+psNrCKpv+2HJXUG1rY9RdK/Je1s+wZJy5B2nxfVWOBQ4GxJnYDlbBcblXQH3sjB8tbAlxf2RlGHOYQQQqiNu+66q95LaGiRw1x/nwC7AadLegqYSMpXBvgxcISkScB4Uhm6RfVzYOvcZfAJYP2y81cD/SQ9TtptfrYG9wwhhBDCIujcuTOdO3eu9zIaVtRhDm0m6jCHEEIItTF8+HAABg4cWNd1dGRN1WHukDvMknaRZEnrLuT1QyVt08T5nSWttwjr6ylp7ybOWdJJhWMrS5oj6fyFvWcIIYQQllzDhw//NGgO7a9DBszAXqSqEXsuzMW2j7d9dxNDdiaVdFtYPYGKAXP2IlDs5Lc7C5Z/qztJkQMfQgghtINx48Yxbty4ei+jYXW4lIzceGMqqYbwTbbXzcd7ACOBFUgPOx5Kygv+P6AfYOBS22dJGg7cYnuUpNOAHYGPgTtJZdtuIZVjm0GqRfwtUim1pYF/AD+2/UGe5708/xeAX+U5HwG+BrwEXG77rML6e+b5JwF/tP24pHH53qvm5iM7AMfm+70F/Mj267lz4JdItZm/BJxt+9w87w3A6qSKF+fkahZI+gnwa+BV4Hlgdr7HKqQGJ1/KSzvS9kP5HquSgv7ptqsG/sv0WMs99ju76t9VCCGEEBY0LR6Yr4tGa429M3CH7eckvS1pY9tPknZ0x9g+OVeH6EpqUb2a7V4AklYsTiTpc8AuwLq5k96Ktt+VdBM5oM7j3rV9cX79O+AnpGYkAD2AbwLrktpTjwIGA4NsF3eRy40A9pT0H1IJt1dJgSqk3fNv5DX9P+BXwFH53LqkXxaWB6ZKusj2HOAA229L6gI8Jmk0qQb0caQGJu8D9wBP5XnOAc6y/aCkLwFjSEE+QF/gm011+QshhBBC7Vx88cUAHHjggXVeSWPqiAHzXkBpW3NEfv8k8BhwaS7bdoPtiZJeBNaUdB5wK2kXt+g9Uue7SyTdStr5raRXDpRXJLW3HlM4d4PtT4BnclvqlroDOAl4nbQzXvRFYGTeNV+atFNdcqvt2cBsSW+Qmo78m1RtY5c8ZnVS174vkLr+vQ0g6Vpg7TxmG2C9eU3+WCE3U4G0c18xWI7GJSGEEELtjRyZQoEImOujQ+UwS1qJlB5xSW64cTSwhyTZvh/YEngFuFLSvrbfATYExgE/BS4pzmf7Y2BTYDR557rKrYcDP7O9AXAi8xp9wPzNPkQL2f6IVPbtqHz/ovOA8/P9Dm7ifnOBpSQNIAXA/XNL7An5mqbW85k8vk/+s5rt9/O5/zax7mG2+9nu16lr92Y/ZwghhBCad/fdd3P33U09XhXaUkfbYd4NuML2waUDku4DvinpX8Arti+WtBywsaTbgI9sj5b0AinwpXBtN6Cr7dty3vE/8qn3SSkPJcuT2ll3JtUufqWZdZZfX80fSDvAbxV2eiE1FyndY78WzNMdeCfnVa8LfCMffxQ4S9Jn85p2BSbnc3cCPwPOAJDUp9AWu0WicUkIIYQQOoKOFjDvBZxWdmw0KX/5EeBoSXOAmcC+wGrAZZJKO+2/Kbt2eeBGSaXd2F/k4yOAiyUdQQrSjwP+BvyTFHA2FwxPAj7OjUqGFx/6K7I9hcrVMYYA10p6JX+uNZq53x3AIbkBytR8DbZfkXRKXvurwDOkBxkBjgAuyNcsBdwPHNLMfUIIIYTQBi688EIADjvssDqvpDF1uCoZoXUkdbM9M5eIu55UKeT6WswdjUtCCCGE2thuu+0AuP322+u8ko6r0apkLPEkzWVeagTAzrantdHthuQmLcuS0jBuaKP7hBBCCGEhRaBcX7HDvBiSNNN2t4W4bqn8oOJiIeowhxBCCC0X9Zfrq+FaY3dEkvpIekTSJEnX5wf1kDRO0in54cafS+or6T5JT0gak0vPlcadJel+SX+XtImk6yQ9n0vile5zQ752Si4RVzo+U9LJkp7K62hNibwQQgghLIJzzjmHc845p97LaFgRMC+eukiamP+U8omvAH5tuzcpXeOEwvgVbW8FnEsqObeb7b7ApcDJhXEf2d6S1MHvRlIpvV7AwFySD1KDk76k7oRHFI4vBzySy9LdD0QhyBBCCKGdjB07lrFjx9Z7GQ0rcpgXT7Ns9ym9kdSdFBTflw9dDlxbGF9qbLIOKQC+K5eh6wS8Vhh3U/45GZhi+7U8/4ukZiZvUbnByVvAR8xr3PIE8J1KC4/GJSGEEELt3XTTTc0PCm0mAuaOodRIRKRAuH+VcaWmJp8wf4OTT1iwwckHksYxrynKHM9LeJ9LlX92bA8DhkHKYW79RwkhhBBCWLxEwLwEsD1D0juStrD9APBj4L4KQ6cCq0jqb/vh3Ehl7VzPuSWqNThZKNG4JIQQQqiNM888E4BBgwbVeSWNKQLmJcd+wJ8kdQVeBPYvH2D7I0m7AefmNI6lgLOp3PykkooNTkIIIYRQXw8//HC9l9DQoqxcaDPRuCSEEEIIS4ooK7eYk2RJVxbeLyXpTUm3NHVdE/NNk7RyjdZ2m6QVazFXCCGEEMKSKFIyFg//BXpJ6mJ7FqkCxSt1XhMAtr+3sNdOfmUGPQffWsvlhBBCCB1OSxqWnHbaaQAMHjy4rZcTKogd5sXH7UDp35i9gL+UTkhaTtKlkh6TNEHSTvl4J0lnSpqcG5ocXpjvcElP5nPr5vGbShqf5xgvaZ18fGBuYnJHbmTy+8K9P92tlrRvvs9TxR3xEEIIIbStiRMnMnHixHovo2HFDvPiYwRwfE7D6E1qOrJFPncMcI/tA3J6xKOS7gb2BdYANrL9saTPFeabbntjSYcBg4D/BzwLbJnHbgOcAuyax/cBNiKVm5sq6TzbL5cmk7R+XsfmtqeX3YvCuKjDHEIIIdTYiBEj6r2EhhYB82LC9iRJPUm7y7eVnd4W2FFSqZbMssCXSDWT/2T74zzH24Vrrss/nwB+kF93By6XtBZgoHNh/FjbMwAkPQN8GXi5cP5bwCjb0yvcq/g5og5zCCGEEDqUCJgXLzcBZwIDgJUKxwXsantqcbBSO79qQWmpMUmxychJwL22d8nB+bgK48uvKa6hVQFw1GEOIYQQauOkk04C4LjjjqvzShpT5DAvXi4FhtqeXHZ8DCknWQCSNsrH7yTVTV4qH6+YJlHQnXkPEw5s5drGAj+UtFIL7xVCCCGEGpk6dSpTp05tfmBoE7HDvBix/W/gnAqnTiI1IJmUg+ZpwPeBS4C18/E5wMXA+U3c4veklIxfAve0cm1TJJ0M3CdpLjCB1gfdIYQQQlgIV111Vb2X0NCicUk7y8FmcQd5hO3TJI0DBtluVacPSX2AVW2X5z2XzvcD9rV9RBNz3Absbfvd1ty7OdG4JIQQQghLiqYal8QOc/ubZbtPDefrA/RjwQcFkbRUDsCbjFoXpdZyCCGEENre8ccfD8DQoUPrvJLGFAHzYkjStsCJwDLAC8D+tmdK2oSUsrEc6SG97wBDgS6SvgmcCnwNWBXoCUyXNIy0c/19Sd2A80gBtoETbY+WNA3ol8vF3QCsTqrEcU6ueoGkmfne3wdmATvZfr2pzxGNS0IIIYTqWtKwpOTll19uflBoMxEwt78ukoqVx0+1PbL0JjcJORbYxvZ/Jf0a+KWk04CRwB62H5O0AvABcDwp2P1Zvn4I0Bf4pu1ZkgYU7nUcMMP2BnnsZyus7wDbb0vqAjwmabTtt0hB+iO2j8mNTQ4EfleD7yOEEEIIzbjsssvqvYSGFgFz+2suJeMbwHrAQ7koxtLAw8A6wGu2HwOw/R5AHlPuptxiu9w2wJ6lN7bfqTDmCEm75NerA2sBbwEfAbfk40+QdrcXEI1LQgghhNDRRMC8+BFwl+295jso9abldZD/28TcVefIu9HbAP1tf5AfRFw2n57jeU+IVqrTDETjkhBCCKEt/OY3vwHg1FNPrfNKGlMEzIufR4ALJH3V9j8kdQW+SGprvaqkTXJKxvKkXOL3geVbOPedwM+AIyGlZJTtMncH3snB8rqk3e6FFo1LQgghhNp466236r2EhhYBc/srz2G+w/bg0hvbb0oaCPxF0jL58LG2n5O0B3Bezi+eRdoNvhcYnOds7tfO35GC8adJu8QnMq+FNsAdpEYok4CppOA9hBBCCHU2bNiwei+hoUUd5tBmog5zCCGEEJYUTdVhjtbYHZikYyRNkTRJ0kRJX29i7HBJu7Xn+kIIIYTQMoMGDWLQoEH1XkbDipSMDkpSf1LN5I1tz87l6pau87JCCCGEsBBmzapU/Cq0lwiYO64ewHTbswFsTweQdDywA9AFGA8c7LK8HEl9gT8C3YDpwEDbr0k6AjgE+Bh4xvaeNCEal4QQQgjVtaZxyQUXXNCGKwnNiZSMjutOYHVJz0m6UNJW+fj5tjex3YsUNH+/eJGkzqRugLvZ7gtcCpycTw8GNrLdmxQ4hxBCCCF0eLHD3EHlVtp9gS2ArYGRkgYD70v6FdAV+BwwBbi5cOk6QC/grtwUpRPwWj43Cbg6t8++odJ9o3FJCCGEUHtHHnkkAGeffXadV9KYImDuwGzPBcYB4yRNBg4GepNaab+c22gvW3aZgCm2+1eYcntgS2BH4DhJ69v+uOye0bgkhBBCCB1KBMwdlKR1gE9sP58P9SHVVu4NTJfUDdgNGFV26VRgFUn9bT+cUzTWBv4OrG77XkkPAnuTcpzfrbaGaFwSQggh1EbsLNdXBMwdVzdSk5MVSQ/p/YOUKvEuMBmYBjxWfpHtj3J5uXMldSf9M3I28BxwVT4m4CzbVYPlEEIIIYSOIhqXhDYTjUtCCCGE2vjpT38KRLWMthSNS8J8JH1B0ghJL0h6RtJtktaWNCs3OCn92TeP7y7pijz+hfy6e70/RwghhNAounTpQpcuXeq9jIYVO8wNRqn0xXjgctt/ysf6AMsDF+Vyc+XXjAKetj0kvz8RWM/27k3da5kea7nHfpFzFUIIIVTSmjrMoe01tcMcOcyNZ2tgTilYBrA9UVLPSoMlfRXoC+xRODwU+Iekr9h+oQ3XGkIIIYRQdxEwN55ewBNVzn1F0sTC+8OBzwITc4k6IJWry+PWB+YLmKMOcwghhFB7Bx10EADDhg2r80oaUwTMoegF232KByTtBFTK21Gl41GHOYQQQqi9lVZaqd5LaGgRMDeeKaT6y60Zv5Gkz9j+BEDSZ4ANSbWZQwghhNDGTj311HovoaFFwNx47gFOkXSg7YsBJG1CapW9ANv/kDQBOJaUu0x+/aTtfzR1o2hcEkIIIYSOIMrKNRinsii7AN/JJeKmAEOAV8k5zIU/R+TLfgKsLekfkl4gdf77ST3WH0IIITSi/fffn/3337/ey2hYscPcwUgy8EfbR+X3g4BupZJwALZfBX5Y4fKKBR5tvwPsU/vVhhBCCKElVl999XovoaFFwNzxzAZ+IOlU29Nbe7GkTsWKGCGEEEKov6FDhzY/KLSZCJg7no9JVSp+ARxTPCFpOHCL7VH5/Uzb3SQNAE4AXgP65JzmvwJfBDoBJ9keKakv8EegGzAdGGj7tWoLmfzKDHoOvrXGHy+EEEJY8kXTkiVLBMwd0wXAJEm/b8U1mwK9bL8kaVfgVdvbw6etsTsD5wE72X5T0h7AycABtV58CCGEEOa3zz4pM/Kqq66q80oaUwTMHZDt9yRdARwBzGrhZY/afim/ngycKel00o70A5J6kZqe3JW6a9OJtCM9n2hcEkIIIdTeOuusU+8lNLQImDuus4EngcsKxz4mV0ZRinqXLpz7b+mF7edy+sX3gFMl3QlcD0yx3b+pm0bjkhBCCKH2jjvuuHovoaFFwNxB2X5b0l9J5d8uzYenAX1J+ck7AZ0rXStpVeBt21dJmgkMBE4DVpHU3/bDOUVjbdtTqq0h6jCHEEIIoSOIOswd2x+AlQvvLwa2kvQo8HUKu8plNgAelTSR9ODg72x/ROoQeLqkp4CJwGZttvIQQgghfGrPPfdkzz33rPcyGlbsMHcwtrsVXr9OoYNffv+NwvDf5OPjgHGFcWOAMRXmnghsWes1hxBCCKFpffr0qfcSGloEzAFYsORcCCGEEBYfgwcPrvcSGlqkZIQQQgghhNCE2GHuwCQtR1kDEmAdYAdSG+zxwMG2XXZdxQYlko4ADiFV23jGdpPJVNG4JIQQQphnUZqV7LrrrgCMHj26VssJrRABc8f2XcoakAB32R6a318JfB+4uXRBMw1KBgNr2J4tacX2/SghhBBC4+rfv8mqrqGNRcDcsVVqQLKrpF+RHgb8HDCFQsBM2oGu1qBkEnC1pBuAGyrdMBqXhBBCCLU3aNCgei+hoUXA3IFVaUDyU6Cf7ZclDQGWLbtMVG9Qsj2pSsaOwHGS1rf9cdk9o3FJCCGEEDqUCJg7sCoNSACmS+pGqqtcXhVjKhUalAB/B1a3fa+kB4G9STnO71a7fzQuCSGEEGpjxx13BOCmm26q80oaUwTMHdsGwBmSPgHmAIcCO5NSNaYBj5VfYPsjSbsB5+ac56VIbbafA67KxwScZbtqsBxCCCGE2vn2t79d7yU0NJUVSAihZvr16+fHH3+83ssIIYQQQmiWpCds96t0LuowNyhJ4+u9hhBCCCGEJUGkZHRQSiUuZPuTSudtb1aDe3SyPbfa+ajDHEIIodEtSu3lou222w6A22+/vSbzhdaJHeZ2Jmk5SbdKekrS05L2kDRN0sr5fD9J4/LrIZKulHSPpOclHViY52hJj0maJOnEfKynpL9LuhB4klTJ4veFawZKOi+/npl/9pB0v6SJeT1b5ON7SZqcj51emGOmpKGS/gZEUcgQQgihHeywww7ssMMO9V5Gw4od5vZXqZnI6U2M7w18A1gOmCDpVlKd5LWATUkP4N0kaUvgX6Q6yv+/vTuPt3M89z/++TZiiCGOoU60IdKjWpSQpMRwbISjSlPaitbQJKfGU+oU5dAWR0tbbc20mqOpoZXSEC01BNtMYoiIIeoXVA2vChoiQYbr98d9b1lW11577ey19pO91/f9euVlrWe4n+vJk3hd+879XNfYiDhS0rrA/cB38lhtTUhKfQ24OSJ+KKkP0C9X1/gxMBR4E7hF0hcj4rocx8yI+H6lYF2H2czMrP6OPPLIokNoap5h7n6PAyMl/VjSjhExt4PjJ0fEgoiYA9xBSpJ3z78eJc0kf4qUQAO8EBEPAETEa8BsSdtKWpuUTN9bNv40YGyuyfyZiHgbGA60RsRruc7ylaT6ywCLgXb7ckbEJRExLCKG9enXv+PfDTMzM7PlnGeYu1k7zUQWsfSHl/JGIuVlTII0q3xmRPyydIekQcA7ZcdPBPYDngaujbKyKBFxV56d/jxwuaSzgLeq3MK71dYtm5mZWf2NHDkSgClTphQcSXNywtzN2mkm8jxp+cOfgS+VnTJK0pmkpRAtwInAAuB0SVdGxDxJHyPVWa5kEnAy8AJwQoV4NgReiohfSVoV2Jq0HOPcvK76TeCrwPmdvVc3LjEzM6uP0aNHFx1CU3PC3P0qNRNZBfg/SScBD5YdPxW4AdgAOD0iXgZelvRp4P5UDIN5wIGk5RIfEhFvSnoS2DQiplaIpwU4XtLCPM7BEfGKpP8hLQERcGNETO7ifZuZmdkyOuSQQzo+yBrGjUs6IOlk0otxi4ElwGER8aCkY4BLImJ+A67ZCgwGLgXmRcRPJV0HjIyI1ep9vUZx4xIzMzPrKdy4ZBlJGgHsBWwdEVsAI4EX8+5jgH6dHK9PJw7/B2lWGUlrAgM6c63uoMR/hszMzBqspaWFlpaWosNoWl6SUd0AYE5EvAeQK1Ug6WhgfeAOSXMiYmdJXwVOIi1huCEiTsjHzgN+DvwHcGx+Me9oYEXS8osj23mJ7ipg/Ty7PI60FnmzPOZqwGTgX4C+wHcjYnIe+8/APcB2wEvAqIhYkGs4H5qv+yxwUETMl/QJUhWMPvncb7fNYks6nvTC4EqkFwZPKbnGHaQ6zF8krY/+J25cYmZmzaZejUrKjRkzpiHjWm08O1jdLcBASc9IukjSTgARcR7wMrBzTpbb6hbvAgwBhkv6Yh6jrW7xNsDrpFrI20fEENIyjwPaufZtwL/nWen9SdUu2rwL7BMRWwM7Az/Lnf0glZe7MCI2I81St71EOCkihkfElsBTwH/m7ecC50bE8HxPAEjanaW1nocAQ3M1DUjl6S6LiK0iomKybGZmZvUzZswYJ80FcsJcRUTMI1WvOBR4DZgoaUyFQ2utW7xrHm+apOn5++B2Lr+YNFM8GlglIp4v2SfgDEkzgCnAx4D18r7nImJ6/vwwMCh/3lzS3ZIeJyXpm+XtI4Cr8+ffllyjplrP5SQdKukhSQ8tnt9RiWkzMzOrxcKFC1m4sL2CWNZoXpLRgbxcohVozcnm14EJZYeJ9pXWLRbwm4j4nxovfxVwLXBq2fYDgHWBoRGxUNLzLK3f/F7JcYtJFTjIMX8xIh7LSX9LB9fuTK3nD0TEJcAlACsN2NhvlJqZmdXBbrvtBkBra2uxgTQpJ8xVSNoEWBIRf8mbhrB0ve7bwOrAHNJa5FrqFt8GTJZ0dkT8XdJawOpVljXcDZwJ/K5se3/g7zlZ3hnYsIbbWR14RVJfUsL9Ut7+AGnZxkTS0o82N1N7reeKXIfZzMysPr7xjW8UHUJTc8Jc3WrA+blKxSLSy3KH5n2XAH+W9Epex9xh3eKIeFLSd4FbcnWJhcB/0c5Lc7kr308r7LoS+KOkh4DppC5+HfkeKbF/gdSee/W8/RjgCknHkuo9z83XvqXWWs9mZmbWWAceeGDRITQ112FucpL6AQsiIiTtD3w1IkbVY2zXYTYzM6uP+fNT24d+/TpV0dY6oVodZs8wd4P2mp8UFMt9EbFdyaahwAW5ysY/gHFFxGVmZmbt23PPPQGvYS6KE+YGK2t+8l5e57xiA68n0r8cLKm0vyxZJiLuBrbs5DX6tFM72szMzBrgiCOOKDqEpuaEufEqNj8ByNUthkXEHEnDgJ9GRIukU4FPkMrFDQR+EhG/yufU0kzkOkmrRsR38jljSBU1jpI0LyJWkzSA9KLfGqQ/B0dExN21NmAhlbyrygscFB4AABj+SURBVI1LzMysN2pUc5JqRo8e3e3XtKVch7nxKjY/qcEWwOdJCfD3Ja1fazMR4CJg35KxRvPhxieQlojcnBuobAlMr7UBS0R0mCybmZlZ/cydO5e5c93foChOmBusE81Pyk2OiAV5RvoOUpJcUzORiHgNmC1pW0lrk5Lpe8vGnwaMzbPZn4mIt6m9AUu73LjEzMys/kaNGsWoUXV5J9+WgZdkdIMqzU8WsfSHlpXLT6vwvTPNRCaSlm48TVq68aHxIuKuPDv9eeBySWcBb1W5jXdrWbfsxiVmZmb1d/TRRxcdQlNzwtxgHTQ/eZ40+/xnUvOQUqMknUlaCtECnAgsoPZmIpOAk/O1TqgQ14bASxHxK0mrAluTlmPU0oClJm5cYmZmVh/77rtvxwdZwzhhbrxqzU9OA/5P0kmkpiKlppIaiWwAnB4RLwMv19pMJCLelPQksGlETK0QVwtwvKSFeZyDI+KVWhqwmJmZWfeaMyfVDFhnnXUKjqQ5uXHJciivK54XEZW6/PUYblxiZmZWHy0tLYDrMDeSG5eYmZmZ9WDHHnts0SE0Nc8w2zKTtEKuplHRSgM2jgFfP6c7QzIzM6u7IuouW/erNsPssnK9iKRvS5qZfx0j6TuSjs77zpZ0e/68q6Qr8ud5kn4o6TFJD0haL29fV9IfJE3Lv7bP20+VdImkW4DLCrpVMzOzpvLqq6/y6quvFh1G03LC3EtIGgqMBbYBtgUOAe4GdsyHDANWk9QX2CHvg1SF44GI2BK4K58HcC5wdkQMJ1XwGF9yuaHAqIj4WuPuyMzMzNrsv//+7L///kWH0bS8hrn32IFUb/kdAEmTSM1OhkpaHXiP1OxkGCmJbivo+D7wp/z5YWC3/HkksGmuxgGwRh4H4PqIWFApCEmHkquA9Flj3frcmZmZWZM78cQTiw6hqTlh7j1UYVuQaj2PBe4DZgA7A58AnsrHLCxparKYpX8mPgKMKE+McwJd3iRl6QXduMTMzKzu9thjj6JDaGpOmHuPu4AJkn5ESp73AQ4C1gKOA8YBjwM/Bx4u7/xXwS3AN4GzACQNiYjpnQnIjUvMzMzq48UXXwRg4MCBBUfSnJww9xIR8YikCaSGJwDjI+JRSWuROv7dHxHvSHqXpeuXqzkauFDSDNKfk7uAwxsQupmZmXXgoIMOAlyHuSguK2cN48YlZmZm9TFlyhQARo4cWXAkvZcbl/Rykj4OXAhsSlp7/Cfg+Ih4v9DAzMzMrC6cKBfLM8w9nNJbeA8CF0fEryX1Ib1090ZEHN/ga7txiZmZ9WrLS9OS2bNnAzB48OCCI+m93Likd9sFeDcifg0QEYuB/wbGSbpd0hYAkh6V9P38+XRJ35DUIqlV0jWSnpZ0ZU7AkTRU0p2SHpZ0s6QBeXurpDMk3Ql8q4gbNjMzazbjxo1j3LhxRYfRtLwko+fbjFQ/+QMR8ZakvwKtwI6SngcWAdvnQ3YArgAGAFvlMV4G7gW2l/QgcD6pOclrkkYDPyRV2gBYMyJ2qhSM6zCbmZnV32mnnVZ0CE3NCXPPJ1K95Urb7wSOBJ4DbgB2k9QPGBQRs/Ks8dSI+BuApOnAIOAfwObArXnCuQ/wSsnYE9sLxnWYzczM6m+nnSrOU1k3ccLc8z1Bal39AUlrAAOBR0md/WYDtwLrkFpfl85Iv1fyua1xiYAnImJEO9dst3FJKddhNjMzq49Zs2YBsMkmmxQcSXPyGuae7zagn6SDAfJLfz8DJkTEW8CLwH7AA6T6y8fRcR3mWcC6kkbkMftK2qxB8ZuZmVkHDjvsMA477LCiw2hanmHu4SIiJO0DXCTpe6Qfgm4ETsqH3A3sGhHzJd0NfJwOEuaIeF/Sl4HzJPUn/Tk5hzSbbWZmZt3sjDPOKDqEpuayctYwblxiZmZmPYXLyllDSPK/UJiZmXWDmTNnMnPmzKLDaFqeYV7OSBoE3ATcA2wLPAb8GjgN+ChwQD70HGAVYAEwNle9GAN8AegHfAK4NiK+k8e9GBiez7kmIk7J2/cEfg7MAR4BBkfEXpJWJZWW+wxpScapETE5X+PzwMrAqhGxS3v34sYlZmbWky0vTUsAWlpaAGhtbS00jt7MrbF7nn8DvkKqZzwN+BqpdvIXSGuTDwb+PSIWSRoJnMHSShlDSLWV3wNmSTo/Il4ETo6IN/JLgbflhibPAL/MYz0n6XclMZwM3B4R4yStCUyVNCXvGwFsERFvNOx3wMzMzD5w1llnFR1CU3PCvHx6LiIeB5D0BHBbfrnvcVKd5P7AbyRtTKrB3Lfk3NsiYm4+90lgQ3KljNxUZAVSw5JNSUtyZkfEc/nc35GbjgC7A1+QdFz+vjKwQf58a3vJshuXmJmZ1d/w4cOLDqGpOWFePpXWRl5S8n0J6ZmdDtwREfvkJRyt7Zy7GFhB0kakcnLDI+JNSRNICbCqxCDgSxEx60MbpW2oUofZjUvMzMzqb/r06QAMGTKk4EiakxPmnqk/8FL+PKaG49cgJblzJa0HfI6UZD8NDJY0KCKeB0aXnHMzcJSko/Ls9lYR8WhngnTjEjMzs/o45phjAK9hLooT5p7pJ6QlGd8Gbu/o4Ih4TNKjpDrKs4F78/YFko4EbpI0B5hactrppBcLZyj1x34e2Kuud2FmZmY1Oeccv0RfJFfJaHKSVouIeTkpvhD4S0ScXY+xXYfZzMzMegrXYe7lJJ0s6QlJMyRNz+uMa3WIpOmk2ef+pKoZZmZmthyZNm0a06ZNKzqMpuUZ5h5O0ghSHeWWiHhP0jrAihHxcg3nrhARi7pw7arnuw6zmZn1VMtTDWZwHebu4DrMvdsAYE5EvAcQEXMAJA0lJdKrkZqSjImIVyS1AvcB2wO3SxpLalayRFI/YBYwmFRC7kJgXWA+cEhEPJ0rbLxBqvX8CHBsd92omZlZs7rggguKDqGpOWHu+W4Bvi/pGWAKMJGUEJ8PjIqI1ySNBn4IjMvnrBkROwFI2hrYCbgD2Bu4OSIWSroEODwi/pKXeFwEtHX1+yQwMiIWd88tmpmZNbfNN9+86BCamhPmHi6/sDcU2BHYmZQw/wDYHLg1vctHH+CVktMmln0eTUqY9wcukrQasB1wdT4fYKWSc65uL1l24xIzM7P6u++++wDYbrvtCo6kOTlh7gVy8toKtOZugP8FPBERI9o5pbTxyPXAmZLWAoaSytStCvwjItqrju7GJWZmZt3opJNOAryGuShOmHs4SZsASyLiL3nTEOApYHdJIyLifkl9gU9GxBPl5+cZ6qnAucCfcvL9lqTnJH0lIq7OJee2iIjHOhObG5eYmZnVxy9/6SJWRXLC3POtBpwvaU1gEfAsaUnEJcB5kvqTnvM5pNJxlUwErgZaSrYdAFws6btAX+AqoFMJs5mZmdXHJptsUnQITc1l5axh3LjEzMysPu68804Adtppp4Ij6b1cVq5JSToVmBcRPy3bfjgwPyIuKyQwMzMz65RTTjkF8BrmojhhbjK52cgv6jhWu41LHn9pLoNOvKEelzIzM2uo5a1RSblLL7206BCamhPmXkbSycDBwIvAa8DDZc1Krpe0OjAPuAH4TUR8Np87CLg+IraosfHJ9cDPuu3mzMzMmtTgwYOLDqGpOWHuRXKSuz+pC98KpE58D+fdpc1KTgWIiKckrShpcETMJtVj/n2uqlFT45MKMbgOs5mZWZ1NmTIFgJEjRxYcSXNywty77AhcGxHzASRdX7JvYuVT+D2wH/AjUsI8GtiE2huffIjrMJuZmdXfD37wA8AJc1GcMPc+7SWp7TUbmUjq6DcJiNwK+zPU3vjEzMzMGuzyyy8vOoSm5oS5d7kLmCDpR6RnuzdQtdJ5RPw/SYuB77F05ngWsG4tjU+qceMSMzOz+hg4cGDRITQ1J8y9SEQ8ImkiMB14Abi7xlMnAmcBG+Vx3pf0ZWpvfGJmZmYNdNNNNwGwxx57FBxJc3Ljkl4izxI/TurKtwj4DXBORCypcOz6wHkR8eVGxuTGJWZmZvXR0tICuA5zI7lxSXNYEBFDACR9FPgt0B84pfSgXDv5ZaDLyXJHdZjNzMysPq666qqiQ2hqnmHuJSTNi4jVSr4PBqYB6wBfBz4PrAysSioP96eI2FzSg8C4tvXJuc7yscDTpNJynyH9YHVqREyWNKZ0rIjYpb2YVhqwcQz4+jn1vlUzM7Nltrw3KLHieIa5CUXEbEkfAT6aN40AtoiIN3KDkjZXkcrKnSJpALB+RDws6Qzg9ogYJ2lNYKqkKeVjdcvNmJmZNbk//vGPAOy9994FR9KcnDD3bir5fGs7Ce7vgVtJSzf2A67O23cHviDpuPx9ZWCDDsZy4xIzM7MG+NnPUmNdJ8zFcMLcS+UlGYuBv+dNFWsnR8RLkl6XtAWpaclhbUMAX4qIWWXjbtPeWHk8Ny4xMzOrs2uuuaboEJqaE+ZeSNK6wC+ACyIicre+aq4CvgP0j4jH87abgaMkHZXH2CoiHu1MHK7DbGZmVh/rrLNO0SE0tY8UHYDVzSqSpkt6ApgC3AKcVuO51wD7k5ZntDmdVKJuhqSZ+buZmZkVYNKkSUyaNKnoMJqWq2RYw7gOs5mZWX24DnPjuUpGN5O0NnBb/vqvpLXEr+Xvn42I95dhzAmkUnA1LWLKlTD+FBGbV9jXLY1LzMzMrD4mT55cdAhNzQlzA0TE60BbE5FTgXkR8dO2/UU2/HDjEjMzs56nf//+RYfQ1Jwwd5M8Q/wGsBXwiKS3KUmk8zrhvSLieUkHA8cBAcyIiIPKxjodGEhqOLKkZPtQ4FJgPnBPyfYxlDQbkVS3xiVAu41LHn9pLoNOvKHzv1lmZmYN0lMbl0ycOBGA0aNHFxxJc3LC3L0+CYyMiMV55vmfSNoMOBnYPiLmSFqrbP9PSC2vx8Y/L0D/NXBURNwp6ayyfW5cYmZm1kNdfPHFgBPmojhh7l5XR8TiDo7ZBbgmIuYAlCWl3wMejIhDy0+S1B9YMyLuzJsuBz5Xcogbl5iZmfVQN954Y9EhNDUnzN2rtOHHIj5c1m/l/F+RlmJUMg0YKmmtCglrtfPKr/0BNy4xMzNb/vXr16/oEJqaE+biPA/sBSBpa2CjvP024FpJZ0fE62XJ8U2khiI3SNo9It5uGywi/iFprqQdIuIe4IBOxOLGJWZmZsuxK664AoADDzyw4EiakxuXFOcPwFqSpgNHAM8A5JfvfgjcKekx4OelJ0XE1cCvgOslrVI25ljgQkn3Aws6EYsbl5iZmS3Hxo8fz/jx44sOo2m5cYk1jBuXmJmZ1cfChQsB6Nu3b8GR9F7VGpd4hrnOJK2dW1RPl/SqpJdKvq+4jGNOkFS1bnI+5jlJj0l6RtJlkj5W5fjxkjZdlnjMzMyse/Xt29fJcoG8hrnOCm5acnxEXCNJwDHAHZI2L+8sKKlPRHyjHhfMY1Ws/OE6zGZmVrSeWne53IQJEwAYM2ZMoXE0K88wd4M8+/tzSXcAP5Z0akm5NiTNbKuNLOlgSTPyTPHlFcY6PY/X7rOL5GzgVXJpOUnzJP1vblQyQlKrpGGSjsi1ndvGHyPp/Pz5QElT8+z4LyX1qTRWHX6LzMzMrIoJEyZ8kDRb93PC3H3ampYc294BJU1LdomILYFvle3/CfBRUtOSJRWGKPcI8Kn8eVVgZkRsk6totLkG2Lfk+2hgoqRP58/bR8QQYDFLK2+0NxaSDpX0kKSHFs+fW0OIZmZm1pHW1lZaW1uLDqNpOWHuPvVoWrJmRBxWocNfe1TyeTGpMseHRMRrwGxJ20paG9gEuBfYFRgKTMuVPHYFBlcbK493SUQMi4hhffq5772ZmZn1fF7D3H0a2bSkPVuR6joDvFslYZ9I6vL3NHBtrrss4DcR8T8Vjq82lpmZmVmv4oS5GM9Tx6Yl5XKyexQwIJ/XkUmkpSAvACeUxDI5x/J3SWsBq0fEC7XepBuXmJmZWW/gJRnFaETTEoCz8nnPAMOBncsrZFQSEW8CTwIbRsTUvO1J4LvALZJmALeSEnAzMzOzpuLGJdYwkt4GZhUdhzXUOsCcooOwhvNz7v38jJuDn3N1G0bEupV2eEmGNdKs9jrmWO8g6SE/497Pz7n38zNuDn7Oy85LMszMzMzMqnDCbGZmZmZWhRNma6RLig7AGs7PuDn4Ofd+fsbNwc95GfmlPzMzMzOzKjzDbGZmZmZWhRNm6zJJe0iaJelZSSdW2C9J5+X9M3KzFutBanjGB+RnO0PSfZK2LCJO65qOnnPJccMlLZb05e6Mz7qulmcsqUXSdElPSLqzu2O0rqvh/9n9Jf1R0mP5OY8tIs6exEsyrEsk9SE1StkN+BuphfdXc+OTtmP2JHUe3BPYBjg3IrYpIFxbBjU+4+2ApyLiTUmfA071M+5ZannOJcfdCrwLXBoR13R3rLZsavy7vCZwH7BHRPxV0kcj4u+FBGzLpMbnfBLQPyJOkLQuqWfCv9bS7KxZeYbZuuqzwLMRMTv/RbsKGFV2zCjgskgeANaU5K6BPUeHzzgi7ssdIwEeAD7ezTFa19XydxnSD79/AJxE9Ty1POOvAZMi4q8ATpZ7pFqecwCrSxKwGvAGsKh7w+xZnDBbV30MeLHk+9/yts4eY8uvzj6//wT+3NCIrBE6fM6SPgbsA/yiG+Oy+qnl7/IngX+R1CrpYUkHd1t0Vi+1POcLgE8DLwOPA9+KiCXdE17P5E5/1lWqsK18nU8tx9jyq+bnJ2lnUsK8Q0Mjskao5TmfA5wQEYvTxJT1MLU84xWAocCuwCrA/ZIeiIhnGh2c1U0tz/k/gOnALsAngFsl3R0RbzU6uJ7KCbN11d+AgSXfP076ibWzx9jyq6bnJ2kLYDzwuYh4vZtis/qp5TkPA67KyfI6wJ6SFkXEdd0TonVRrf+/nhMR7wDvSLoL2JK0JtZ6hlqe81jgR5FeZHtW0nPAp4Cp3RNiz+MlGdZV04CNJW0kaUVgf+D6smOuBw7O1TK2BeZGxCvdHagtsw6fsaQNgEnAQZ6J6rE6fM4RsVFEDIqIQcA1wJFOlnuUWv5/PRnYUdIKkvqRXtR+qpvjtK6p5Tn/lfSvCEhaD9gEmN2tUfYwnmG2LomIRZK+CdwM9CG9Nf+EpMPz/l8AN5IqZDwLzCf9ZGs9RI3P+PvA2sBFefZxUUQMKypm67wan7P1YLU844h4StJNwAxgCTA+ImYWF7V1Vo1/l08HJkh6nLSE44SImFNY0D2Ay8qZmZmZmVXhJRlmZmZmZlU4YTYzMzMzq8IJs5mZmZlZFU6YzczMzMyqcMJsZmZmZlaFE2YzsyYnabGk6SW/BnVxvCGS9iz5/gVJJ3Y1zjzW0ZKeknRlB8fNy/8dJKmhZdEk3ShpzQrbT5V0XCOvbWbdw3WYzcxsQUQMqbRDqbC2ImJJJ8YbQuoKeCNARFzPPzdOWFZHkrpJPlen8bosIvbs+Cgz68k8w2xmZh+SZ2WfknQR8AgwUNLFkh6S9ISk00qOHS7pPkmPSZoqqT/wv8DoPFs9WtIYSRfk4zeUdJukGfm/G+TtEySdl8eaLenLFeL6BTAYuF7Sf5fP4EqaWW12XNLdkoaUfL83t3QvPWaMpMmSbpI0S9IpJfuuk/Rw/j04tGT785LWyZ9PzudNIXVPM7NewAmzmZmtUrIc49q8bRPgsojYKiJeAE7O3Ru3AHaStEVuuzsR+FZEbAmMBN4hdX6cGBFDImJi2bUuyONuAVwJnFeybwCwA7AX8KPyICPicOBlYOeIOHsZ7nM8MAZA0ieBlSJiRoXjPgscQJop/4qktq6V4yJiKGn2/GhJa5eeJGkoqQ3xVsC+wPBliNHMlkNOmM3MbEFObodExD552wsR8UDJMftJegR4FNgM2JSUVL8SEdMAIuKtiFjUwbVGAL/Nny8nJchtrouIJRHxJLBeF++pkquBvST1BcYBE9o57taIeD0iFgCTSmI8WtJjwAPAQGDjsvN2BK6NiPkR8Rb1W4ZiZgXzGmYzM6vknbYPkjYCjgOGR8SbkiYAKwMCoovXKT3/vZLPquHcRXx44mflqheKmC/pVmAUsB9pprijmABCUgtpBn1EHqe1net19ffDzJZDnmE2M7OOrEFKoOdKWg/4XN7+NLC+pOEAklaXtALwNrB6O2PdR1q2AGnZwz1diOt5YOt87a2BjWo4ZzxpGci0iHijnWN2k7SWpFWALwL3Av2BN3Oy/Clg2wrn3QXsI2kVSasDe3fqbsxsueUZZjMzqyoiHpP0KPAEMJuUQBIR70saDZyfk8sFpFnYO4ATJU0Hziwb7mjgUknHA68BY7sQ2h+Ag/N1pgHP1HAvD0t6C/h1lcPuIS0X+TfgtxHxkKTHgcMlzQBmkZZllI/9iKSJwHTgBeDuzt6QmS2fFOF/PTIzs+YgaX2gFfhUpVJ5ksYAwyLim90cmpktx7wkw8zMmoKkg4EHSRU/OlNX2syanGeYzczMzMyq8AyzmZmZmVkVTpjNzMzMzKpwwmxmZmZmVoUTZjMzMzOzKpwwm5mZmZlV4YTZzMzMzKyK/w8FfjK2y1KjnQAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "paid_fraction.sort_values().plot(kind='barh', figsize=(10, 8))\n", "overall_payment_ratio = loans_50_jobs.loan_status.value_counts(normalize=True)['Fully Paid']\n", "plt.vlines(overall_payment_ratio, 0, 50, linestyle=':')\n", "plt.xlabel(\"Fraction fully paid\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This plot shows that there seems to be some information in the employment title. You might also note that some of the titles have capitalized an uncapitalized variants. In principle, the spelling difference could contain additional information. However, at least when looking directly at the outcomes, the different capitalization variants largely have the same outcomes (we could do a more rigorous statistical analysis for this if we were interested). Consolidating the capitalization could be done simply by lowercasing everything:" ] }, { "cell_type": "code", "execution_count": 62, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "C:\\Users\\t3kci\\anaconda3\\lib\\site-packages\\ipykernel_launcher.py:1: SettingWithCopyWarning: \n", "A value is trying to be set on a copy of a slice from a DataFrame.\n", "Try using .loc[row_indexer,col_indexer] = value instead\n", "\n", "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n", " \"\"\"Entry point for launching an IPython kernel.\n" ] } ], "source": [ "loans_paid['emp_title_lower'] = loans_paid['emp_title'].str.lower()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If you explore the dataset more, you'll find that there are also misspellings, which are a bit harder to correct and which we'll leave for now.\n", "After we consolidated the capitalization we can replace all but the 50 most common titles by \"other\":" ] }, { "cell_type": "code", "execution_count": 63, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "C:\\Users\\t3kci\\anaconda3\\lib\\site-packages\\ipykernel_launcher.py:3: SettingWithCopyWarning: \n", "A value is trying to be set on a copy of a slice from a DataFrame.\n", "Try using .loc[row_indexer,col_indexer] = value instead\n", "\n", "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n", " This is separate from the ipykernel package so we can avoid doing imports until\n" ] }, { "data": { "text/plain": [ "other 297838\n", "manager 8162\n", "teacher 8071\n", "owner 5769\n", "driver 4104\n", "registered nurse 3850\n", "supervisor 3569\n", "sales 3486\n", "rn 3169\n", "project manager 2444\n", "general manager 2268\n", "office manager 2265\n", "truck driver 2037\n", "director 1788\n", "president 1682\n", "engineer 1630\n", "sales manager 1542\n", "operations manager 1388\n", "police officer 1319\n", "nurse 1268\n", "accountant 1229\n", "vice president 1220\n", "store manager 1215\n", "technician 1153\n", "mechanic 1108\n", "account manager 1051\n", "administrative assistant 1040\n", "attorney 984\n", "analyst 962\n", "server 923\n", "branch manager 871\n", "assistant manager 868\n", "executive assistant 832\n", "paralegal 779\n", "foreman 763\n", "electrician 756\n", "software engineer 747\n", "customer service 745\n", "operator 745\n", "supervisor 722\n", "clerk 682\n", "controller 669\n", "machine operator 667\n", "ceo 666\n", "consultant 663\n", "program manager 642\n", "administrator 634\n", "it manager 551\n", "manager 549\n", "business analyst 548\n", "principal 548\n", "Name: emp_title_50, dtype: int64" ] }, "execution_count": 63, "metadata": {}, "output_type": "execute_result" } ], "source": [ "title_counts_lower = loans_paid.emp_title_lower.value_counts()\n", "common_titles = title_counts_lower.index[:50]\n", "loans_paid['emp_title_50'] = loans_paid['emp_title_lower'].map(lambda x: x if x in common_titles else 'other')\n", "loans_paid['emp_title_50'].value_counts()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now, we have a relatively clean categorical variable, with only 51 values, which we can easily one-hot encode. TODO better transition" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Impact encoding\n", "There is one more encoding scheme for categorical data that is commonly used and that recently has increased in popularity, known as impact encoding or target encoding.\n", "This method of encoding categorical variables uses information about the classification target and is particularly useful for categorical features with many categories.\n", "\n", "The idea behind impact encoding is to replace the categorical feature by the average outcome of the respective category.\n", "In our case of employment title, the feature would become something like \"likelihood of fully paying based on job title alone\", i.e. the value for the job title in Figure TODO. In other words, if we want to encode a value of, say, 'ceo', we look at the average frequency of of loans with 'emp_title' ceo to be fully paid:" ] }, { "cell_type": "code", "execution_count": 64, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0.7534013605442177" ] }, "execution_count": 64, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# should we recompute with lower-cased here? Or get rid of lower-casing?\n", "paid_fraction['CEO']" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "So any time we would an 'emp_title' of 'CEO' our new feature (which will take the place of 'emp_title') will be 0.75...\n", "\n", "In principle we could compute this new feature using pandas alone by first computing the paid fraction (redoing it for all titles, not just the top 50, and taking lower-casing into account):" ] }, { "cell_type": "code", "execution_count": 65, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "C:\\Users\\t3kci\\anaconda3\\lib\\site-packages\\pandas\\core\\generic.py:6245: SettingWithCopyWarning: \n", "A value is trying to be set on a copy of a slice from a DataFrame\n", "\n", "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n", " self._update_inplace(new_data)\n" ] } ], "source": [ "# Fill in emp_title_lower with \"missing\" where it's missing (otherwise these will be ignored)\n", "# inplace means we're overwriting the existing column\n", "loans_paid.emp_title_lower.fillna('missing', inplace=True)\n", "# redoing the pandas trickery from above\n", "paid_fraction_full = loans_paid.groupby('emp_title_lower').loan_status.value_counts(normalize=True).unstack('loan_status')['Fully Paid']\n", "# NaN means none were paid\n", "paid_fraction_full.fillna(0, inplace=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "And then adding the new feature based on ``emp_title`` to each row:" ] }, { "cell_type": "code", "execution_count": 66, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "emp_title_lower\n", " \\toffice manager/medical assistant 0.0\n", " \\tsecurity guard 1.0\n", " ag 1.0\n", " assembler 0.0\n", " diversion investigator 1.0\n", " ... \n", "zoo keeper 0.0\n", "zoo lab coordinator 0.0\n", "zookeeper 1.0\n", "zpic coordinator 1.0\n", "| principal business solution architect| 1.0\n", "Name: Fully Paid, Length: 92041, dtype: float64" ] }, "execution_count": 66, "metadata": {}, "output_type": "execute_result" } ], "source": [ "paid_fraction_full" ] }, { "cell_type": "code", "execution_count": 67, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "emp_title_lower\n", "supervisor 0.722992\n", "assistant to the treasurer (payroll) 1.000000\n", "teacher 0.784661\n", "accounts examiner iii 1.000000\n", "senior director risk management 1.000000\n", " ... \n", "systems engineer 0.843284\n", "accounting 0.782178\n", "account rep 0.833333\n", "lpn 0.715953\n", "equipment maint supervisor 1.000000\n", "Name: Fully Paid, Length: 383181, dtype: float64" ] }, "execution_count": 67, "metadata": {}, "output_type": "execute_result" } ], "source": [ "paid_fraction_full[loans_paid.emp_title_lower]" ] }, { "cell_type": "code", "execution_count": 68, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "100 supervisor \n", "152 assistant to the treasurer (payroll)\n", "170 teacher\n", "186 accounts examiner iii\n", "215 senior director risk management\n", " ... \n", "999995 systems engineer\n", "999996 accounting\n", "999997 account rep\n", "999998 lpn\n", "999999 equipment maint supervisor\n", "Name: emp_title_lower, Length: 383181, dtype: object" ] }, "execution_count": 68, "metadata": {}, "output_type": "execute_result" } ], "source": [ "loans_paid.emp_title_lower" ] }, { "cell_type": "code", "execution_count": 69, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "emp_title_lower\n", " \\toffice manager/medical assistant 0.0\n", " \\tsecurity guard 1.0\n", " ag 1.0\n", " assembler 0.0\n", " diversion investigator 1.0\n", " ... \n", "zoo keeper 0.0\n", "zoo lab coordinator 0.0\n", "zookeeper 1.0\n", "zpic coordinator 1.0\n", "| principal business solution architect| 1.0\n", "Name: Fully Paid, Length: 92041, dtype: float64" ] }, "execution_count": 69, "metadata": {}, "output_type": "execute_result" } ], "source": [ "paid_fraction_full" ] }, { "cell_type": "code", "execution_count": 70, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
emp_title_lowerFully Paidloan_status
100supervisor0.722992Fully Paid
152assistant to the treasurer (payroll)1.000000Fully Paid
170teacher0.784661Fully Paid
186accounts examiner iii1.000000Fully Paid
215senior director risk management1.000000Fully Paid
269front office lead0.600000Fully Paid
271sewell collision center1.000000Fully Paid
296manager0.759863Fully Paid
369service advisor0.735294Fully Paid
379stylist0.738739Fully Paid
\n", "
" ], "text/plain": [ " emp_title_lower Fully Paid loan_status\n", "100 supervisor 0.722992 Fully Paid\n", "152 assistant to the treasurer (payroll) 1.000000 Fully Paid\n", "170 teacher 0.784661 Fully Paid\n", "186 accounts examiner iii 1.000000 Fully Paid\n", "215 senior director risk management 1.000000 Fully Paid\n", "269 front office lead 0.600000 Fully Paid\n", "271 sewell collision center 1.000000 Fully Paid\n", "296 manager 0.759863 Fully Paid\n", "369 service advisor 0.735294 Fully Paid\n", "379 stylist 0.738739 Fully Paid" ] }, "execution_count": 70, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# we use join to map 'emp_title_lower' to the faction using the content of paid_fraction_full\n", "loans_paid_new = loans_paid.join(paid_fraction_full, 'emp_title_lower')\n", "# show the result; the new column is somewhat confusingly called 'Fully Paid' and we should probably rename it\n", "loans_paid_new[['emp_title_lower', 'Fully Paid', 'loan_status']].head(10)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "However, there's two caveats to this (relatively) simple approach: for a classification task like this, it might be beneficial to encode using the *log-odds* of the target (which is a simple non-linear transformation) instead of using the frequency (it will become clear why in Chapter TODO). Also, if a category appears only rarely, of just once, then the feature will be deceptively informative. If there is just one sample, the feature will encode just the target of this row. This is a typical case of information leakage. The resulting model is likely to learn that this column is particularly useful, though this knowledge will not translate to unseen data.\n", "```{margin}\n", "You can install the category_encoders package by running ``pip install category_encoders`` on the command line.\n", "```\n", "There are a couple of work-arounds for this, that we won't discuss in detail here, that either use additional hold-out sets or smoothing. Many of them are implemented in the ``category_encoders`` package, from which I'll particulary recomment the TargetEncoder (which implements todo) and GLMMEncoder (which implements TODO), which was shown to work well in practice todo reference masters thesis.\n", "The ``category_encoder`` package broadly follows the API of scikit-learn, but also allows you to specify which columns the encoding should be applied to. So you can either use this feature, or the ColumnTransformer." ] }, { "cell_type": "code", "execution_count": 78, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
emp_title_lower
1000.722992
1520.775412
1700.784661
1860.939599
2150.775412
2690.603155
2710.775412
2960.759863
3690.735294
3790.738739
\n", "
" ], "text/plain": [ " emp_title_lower\n", "100 0.722992\n", "152 0.775412\n", "170 0.784661\n", "186 0.939599\n", "215 0.775412\n", "269 0.603155\n", "271 0.775412\n", "296 0.759863\n", "369 0.735294\n", "379 0.738739" ] }, "execution_count": 78, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# TargetEncoder implements a smoothing variant from \n", "from category_encoders import TargetEncoder\n", "te = TargetEncoder().fit(loans_paid[['emp_title_lower']], loans_paid.loan_status == 'Fully Paid')\n", "te.transform(loans_paid[['emp_title_lower']].head(10))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You can see that for categories with many samples, such as supervisor and teacher, the feature is identical to what we computed above, whereas for those with less features, such as 'sewell collision center', the estimate is much lower.\n", "\n", "```{margin} TODO\n", "maybe do a classification example with this?\n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Summary\n", "Categorical variables are quite common in many different settings, and while some models might be able to deal with them natively, preprocessing is necessary for most machine learning models. Using one-hot-encoding is the most common method and usually a good place to start. If there are many categories, it might be useful to encode only the most common ones, and combine all the infrequent ones.\n", "Another common approach to dealing with categorical variables with many categories is target encoding, which is implemented in several variants in the ``category_encoders`` package." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.6" } }, "nbformat": 4, "nbformat_minor": 4 }