214 lines
18 KiB
Plaintext
214 lines
18 KiB
Plaintext
{
|
||
"cells": [
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"# 信息熵\n",
|
||
"\n",
|
||
"\n",
|
||
"## 信息量\n",
|
||
"\n",
|
||
"$$\n",
|
||
"I_i=\\log_2(\\frac{1}{p_i})=-\\log_2{p_i}\n",
|
||
"$$\n",
|
||
"\n",
|
||
"其中，$I_i$ 为$i$的信息量，$p_i$ 为$i$出现的概率。显然，当$i$出现的机率越小的时候它的信息量就越大。\n",
|
||
"\n",
|
||
"## 信息熵\n",
|
||
"\n",
|
||
"$$\n",
|
||
"H(X)=\\sum_{i=1}^{n}(p_i \\times \\log_2(\\frac{1}{p_i}))\n",
|
||
"$$\n",
|
||
"\n",
|
||
"信息熵：信息量的期望\n",
|
||
"\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"设有 __模型1__ 和 __模型2__ 对选情进行预测：\n",
|
||
"\n",
|
||
"__模型1__\n",
|
||
"\n",
|
||
"| Prediction | Ground truth | Correct |\n",
|
||
"|-------------|--------------|----------|\n",
|
||
"| 0.3 0.3 0.4 | 0 0 1 | yes |\n",
|
||
"| 0.3 0.4 0.3 | 0 1 0 | yes |\n",
|
||
"| 0.1 0.2 0.7 | 1 0 0 | no |\n",
|
||
"\n",
|
||
"__模型2__\n",
|
||
"\n",
|
||
"| Prediction | Ground truth | Correct |\n",
|
||
"|-------------|--------------|----------|\n",
|
||
"| 0.1 0.2 0.7 | 0 0 1 | yes |\n",
|
||
"| 0.1 0.7 0.2 | 0 1 0 | yes |\n",
|
||
"| 0.3 0.4 0.3 | 1 0 0 | no |\n",
|
||
"\n",
|
||
"从 Correct 列来看(accurary)，两个模型的表现都是2对1对，但从具体的 prediction 来看，显然 __模型2__ 的表现更好。\n",
|
||
"\n",
|
||
"\n",
|
||
"## Classification Error（分类错误率）\n",
|
||
"\n",
|
||
"$$\n",
|
||
"\\text{classification error} = \\frac{\\text{count of error items}}{\\text{count of all items}}\n",
|
||
"$$\n",
|
||
"\n",
|
||
"显然，按 accuracy 来判断模型的优质程序不够精细。\n",
|
||
"\n",
|
||
"## Mean Squared Error （均方误差）\n",
|
||
"\n",
|
||
"$$\n",
|
||
"MSE = \\frac{1}{n}\\sum_{i}^{n}(\\hat{y_i} - y_i)^2\n",
|
||
"$$\n",
|
||
"\n",
|
||
"\n",
|
||
"__模型一__:\n",
|
||
"\n",
|
||
"$$\n",
|
||
"loss_1 = (0.3-0)^2 + (0.3-0)^2 + (0.4-1)^2 = 0.54\\\\\n",
|
||
"loss_2 = (0.3-0)^2 + (0.4-1)^2 + (0.3-0)^2 = 0.54\\\\\n",
|
||
"loss_3 = (0.1-1)^2 + (0.2-0)^2 + (0.7-0)^2 = 1.32\\\\\n",
|
||
"L = \\frac{0.54 + 0.54 + 1.32}{3} = 0.8\n",
|
||
"$$\n",
|
||
"\n",
|
||
"\n",
|
||
"__模型二__:\n",
|
||
"\n",
|
||
"$$\n",
|
||
"loss_1 = (0.1-0)^2 + (0.2-0)^2 + (0.7-1)^2 = 0.138\\\\\n",
|
||
"loss_2 = (0.1-0)^2 + (0.7-1)^2 + (0.2-0)^2 = 0.138\\\\\n",
|
||
"loss_3 = (0.3-1)^2 + (0.4-0)^2 + (0.3-0)^2 = 0.72\\\\\n",
|
||
"L = \\frac{0.138 + 0.138 + 0.72}{3} = 0.332\n",
|
||
"$$\n",
|
||
"\n",
|
||
"显然，MSE 可以更好地反映出模型的表现差异。但它的问题在训练时开始阶段的梯度下降速率非常慢。\n",
|
||
"\n",
|
||
"\n",
|
||
"## Cross Entropy Error Function （交叉熵损失函数）\n",
|
||
"\n",
|
||
"### 二分类\n",
|
||
"\n",
|
||
"$$\n",
|
||
"L = \\frac{1}{N}\\sum_{i}L_i = \\frac{1}{N}\\sum_i-[y_i\\log(p_i)+(1-y_i)\\log(1-p_i)]\n",
|
||
"$$\n",
|
||
"\n",
|
||
"* $y_i$: 样本$i$的label, 正类为1, 负类为0\n",
|
||
"* $p_i$: 样本$i$预测为正的概率\n",
|
||
"\n",
|
||
"### 多分类\n",
|
||
"\n",
|
||
"$$\n",
|
||
"L = \\frac{1}{N}\\sum_{i}L_i = \\frac{1}{N}\\sum_i -\\sum_{c=1}^{M}y_{ic}\\log(p_{ic})\n",
|
||
"$$\n",
|
||
"\n",
|
||
"* $M$： 类别的数量 \n",
|
||
"* $y_{ic}： 指示变量（0或1），如果该类别和样本$i$的类别相同就是1,否则是0\n",
|
||
"* $p_{ic}： 对于难测样本$i$属于类别$c$的预测查概率\n",
|
||
"\n",
|
||
"__模型一__:\n",
|
||
"\n",
|
||
"$$\n",
|
||
"loss_1 = -(0 \\times \\log 0.3 + 0 \\times \\log 0.3 + 1 \\times \\log 0.4) = 0.91\\\\\n",
|
||
"loss_2 = -(0 \\times \\log 0.3 + 1 \\times \\log 0.4 + 0 \\times \\log 0.3) = 0.91\\\\\n",
|
||
"loss_3 = -(1 \\times \\log 0.1 + 0 \\times \\log 0.2 + 0 \\times \\log 0.7) = 2.30\\\\\n",
|
||
"L = \\frac{0.91 + 0.91 + 2.30}{3}=1.37\n",
|
||
"$$\n",
|
||
"\n",
|
||
"\n",
|
||
"__模型二__:\n",
|
||
"\n",
|
||
"$$\n",
|
||
"loss_1 = -(0 \\times \\log 0.1 + 0 \\times \\log 0.2 + 1 \\times \\log 0.7) = 0.35\\\\\n",
|
||
"loss_2 = -(0 \\times \\log 0.1 + 1 \\times \\log 0.7 + 0 \\times \\log 0.2) = 0.35\\\\\n",
|
||
"loss_3 = -(1 \\times \\log 0.3 + 0 \\times \\log 0.4 + 0 \\times \\log 0.4) = 1.20\\\\\n",
|
||
"L = \\frac{0.35 + 0.35 + 1.20}{3}=0.63\n",
|
||
"$$\n",
|
||
"\n",
|
||
"可以发现，交叉熵损失函数也可以反映出 __模型一__ 和 __模型二__ 的优劣\n",
|
||
"\n",
|
||
"### 函数性质"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 1,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stderr",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"<ipython-input-1-f5324e002a15>:6: RuntimeWarning: divide by zero encountered in log\n",
|
||
" y = -np.log(x) # 二分类化简\n"
|
||
]
|
||
},
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"Text(0, 0.5, 'loss')"
|
||
]
|
||
},
|
||
"execution_count": 1,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
},
|
||
{
|
||
"data": {
|
||
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXgAAAEGCAYAAABvtY4XAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy86wFpkAAAACXBIWXMAAAsTAAALEwEAmpwYAAAjLElEQVR4nO3dd3xd5X3H8c9P29rWtmXJ8pAHGMsbmxHMCKGElSYhQCChodDQkNKsNqtN05k2TdqmtBAHUgIhhkIScBgBggGbYWN57y0vyZq2pofG0z/OlSJcDxnr6tx77vf9et2Xru49957fI9lfnfuc5zyPOecQEZHgifO7ABERCQ8FvIhIQCngRUQCSgEvIhJQCngRkYBK8LuA/vLy8lxZWZnfZYiIRI2VK1c2OOfyT/ZcRAV8WVkZlZWVfpchIhI1zGzPqZ5TF42ISEAp4EVEAkoBLyISUAp4EZGAUsCLiASUAl5EJKAU8CIiARX1Ad/T43hg8XaWbKv3uxQRkYgS9QEfF2f8eMkuXttc63cpIiIRJeoDHqAoM4XalmN+lyEiElECEfCFmSkcbDnqdxkiIhElMAFfq4AXEXmfQAR8UVYyda3H6OnR+rIiIr0CEfCFmSl09zga2tUPLyLSKzABD1DbrIAXEekViIAv6g149cOLiPQJRMD3HsFrJI2IyO8FIuDz0pOIM6hTwIuI9AlEwCfEx5GfkawjeBGRfgIR8NB7sZNOsoqI9ApUwKuLRkTk9wIT8EWarkBE5H0CE/CFmckc7ujkaGe336WIiESEAAW8N1SyTv3wIiJAgAK+KEtj4UVE+gtMwOtiJxGR9wtcwGskjYiIJzABn5mSwLDEeA42K+BFRCBAAW9mFGbqalYRkV6BCXjovdhJo2hERCBgAV+UpYudRER6hT3gzSzezFab2fPh3lfv4tvOaek+EZGhOIK/H9g8BPuhMDOF4109NB/pHIrdiYhEtLAGvJmNAj4KPBzO/fQq0lh4EZE+4T6C/3fgL4CeU21gZveYWaWZVdbX15/TzgozkwE0VFJEhDAGvJldB9Q551aebjvn3ALn3Czn3Kz8/Pxz2qfmoxER+b1wHsFfDNxgZlXAk8AVZvbzMO6Pgt4jeHXRiIiEL+Cdc99wzo1yzpUBtwCLnXO3h2t/AMkJ8eSkJSngRUQI2Dh4gJLhw9jT2O53GSIivhuSgHfOveGcu24o9jWpKJPNNa0aCy8iMS9wR/CTRmTQ1H6c+jadaBWR2Ba4gJ9YlAHA1oOtPlciIuKvwAX8pKJMALbUKOBFJLYFLuBz0pIoyEhmi47gRSTGBS7gASaNyGTLwRa/yxAR8VUwA74og+11bXR1n3KGBBGRwAtswB/v6qFK4+FFJIYFMuB7R9Js1olWEYlhgQz48QXpxMeZhkqKSEwLZMAnJ8QzLj9NJ1pFJKYFMuABJoamLBARiVWBDfhJRRkcOHyElqNavk9EYlOgAx5gm/rhRSRGBTfgR4SmLFDAi0iMCmzAj8xKISMlQSdaRSRmBTbgzYzJRZlsOKCAF5HYFNiAB5hVNpwNB5ppP9bldykiIkMu0AE/b1wuXT2Oyj2H/C5FRGTIBTrgZ44eTkKcsWxXo9+liIgMuUAHfGpSAhUl2Qp4EYlJgQ54gHljc1m3v5k29cOLSIwJfMDPHZtLd4+jsqrJ71JERIZU4AN+xuhsEuONZbsU8CISWwIf8KlJCVSMyuZd9cOLSIwJfMCDN1xyw4FmWjXxmIjEkJgI+L5+eI2HF5EYEhMBP6N0eKgfXt00IhI7YiLghyXFM71kOO/uVMCLSOyIiYAHuGxiPuv2N3Ow+ajfpYiIDImYCfiPnF8EwMsbD/pciYjI0IiZgB9fkE55QTovbajxuxQRkSERMwEP8AdTinhvdxONbcf8LkVEJOxiKuCvmTKCHgevbKr1uxQRkbCLqYCfPCKD0bmp/HaD+uFFJPjCFvBmlmJm75nZWjPbaGbfDde+zqImrjm/iHd2NtB8RFe1ikiwhfMI/hhwhXOuApgGXGNmc8O4vwG5ZkoRnd2O1zarm0ZEgi1sAe88baFvE0M3F679DVTFqGxGZKXwkrppRCTgwtoHb2bxZrYGqANedc4tP8k295hZpZlV1tfXh7McAOLijGumFPHmtnqaO9RNIyLBFdaAd851O+emAaOAOWY25STbLHDOzXLOzcrPzw9nOX0+MXMUx7t6+PXq/UOyPxERPwzJKBrn3GHgdeCaodjfmZw/Moupo7J4csU+nPO910hEJCzCOYom38yyQ/eHAR8GtoRrf2fr1jmlbDnYyup9h/0uRUQkLMJ5BD8CeN3M1gEr8Prgnw/j/s7K9RUjSU2K58n39vpdiohIWIRzFM0659x059xU59wU59zfhmtfH0R6cgI3VIzkN2trtNKTiARSTF3JeqJb55RypLOb59ZU+12KiMigi+mAnzoqi8kjMlmobhoRCaCYDngz47YLS9lY3cJ7u5v8LkdEZFDFdMADfGLGKHLSknjozZ1+lyIiMqhiPuCHJcVz50VlLN5Sx5aDLX6XIyIyaGI+4AE+M280qUnx/PjNXX6XIiIyaBTwQHZqErfNKWXR2mr2NXX4XY6IyKBQwIfcdekY4gweXqqjeBEJBgV8yIisYXxsejFPrthHfavWbBWR6KeA7+fe+ePp6nE8sHi736WIiJwzBXw/Y/LSuGV2CU8s30tVQ7vf5YiInBMF/Anuv7KcxPg4/vWVrX6XIiJyThTwJyjITOHuS8fw/Loa1u0/7Hc5IiIfmAL+JO7+0Fhy0pL43ktbtCCIiEQtBfxJZKQk8sUrxvPOzkYWb6nzuxwRkQ9EAX8Kn75wNOML0vnOoo0cOd7tdzkiImdNAX8KSQlx/N2NU9h/6AgPvK5hkyISfRTwpzFvXC5/OL2YBUt2saOu1e9yRETOyoAC3szuN7NM8zxiZqvM7OpwFxcJvvnRyQxLjOfbz27QCVcRiSoDPYL/nHOuBbgaGA7cAXwvbFVFkLz0ZP7imkks29XEMyv3+12OiMiADTTgLfT1WuBx59zGfo8F3m1zSpldNpy//c0mDhw+4nc5IiIDMtCAX2lmr+AF/MtmlgH0hK+syBIXZ/zgk9Podo6/eGYtPT3qqhGRyDfQgL8L+Dow2znXASQCfxS2qiJQaW4q3/7oeby9o5HHl+3xuxwRkTMaaMDPA7Y65w6b2e3At4Hm8JUVmW6dU8L8ifn800ub2Vnf5nc5IiKnNdCAfxDoMLMK4CvATuCxsFUVocyMf/n4VFIS4/niL1ZztFMXQIlI5BpowHc5b4zgjcADzrn/AjLCV1bkKshM4Yc3V7CppoXv/maT3+WIiJzSQAO+1cy+gTc88gUzi8Prh49JV0wq5POXjWPhe3t5dvUBv8sRETmpgQb8p4BjeOPhDwKjgO+Hraoo8NWrJzCnLIdv/nq9rnIVkYg0oIAPhfoTQJaZXQccdc7FXB98fwnxcfzo1ukMS4znnsdW0tzR6XdJIiLvM9CpCm4G3gM+CdwMLDezT4SzsGhQlJXCg7fPZN+hDu5buIqu7pi5NEBEosBAu2i+hTcG/rPOuc8Ac4C/Cl9Z0WPOmBz+4aYLWLq9gb9/YbPf5YiI9EkY4HZxzrn+K180opko+9w8u4Rtta08/NZuxhekc/vc0X6XJCIy4ID/rZm9DCwMff8p4MXwlBSdvnHtZHY1tPPXz22gICOZq88v8rskEYlxAz3J+jVgATA1dFvgnPvLcBYWbeLjjAdum84Fo7L54sLVVFY1+V2SiMS4AXezOOd+6Zz7cuj26zNtb2YlZva6mW0ys41mdv+5lRr5UpMS+J87Z1OcPYy7flbJtloNnxQR/5w24M2s1cxaTnJrNbOWM7x3F/AV59x5wFzgC2Z23mAVHqly0pL42efmkJwQxx2PLGdPY7vfJYlIjDptwDvnMpxzmSe5ZTjnMs/w2hrn3KrQ/VZgM1A8eKVHrpKcVB6/60KOd/Vw20+Ws6+pw++SRCQGDclIGDMrA6YDy0/y3D1mVmlmlfX19UNRzpCYWJTB43ddSOvRTm57eBnVWihERIZY2APezNKBXwJ/Hlr2732ccwucc7Occ7Py8/PDXc6QmlKcxeN3Xcjh9k5u+8kyrQYlIkMqrAFvZol44f6Ec+5X4dxXpKooyeZnd82hsf04Nz/0LlUN6pMXkaERtoA3MwMeATY7534Yrv1Egxmlw1l491yOdHZz84/fZbtG14jIEAjnEfzFeNMLX2Fma0K3a8O4v4g2pTiLp+6ZC8DNP36X1XsP+VyRiARd2ALeOfeWc86cc1Odc9NCt5i++rW8MIOnPz+PzGGJ3PaT5SzeUut3SSISYJpPZoiNzk3jmc9fxPiCdO5+bCVPrdjrd0kiElAKeB/kZyTz5D1zuXh8Hn/5y/X8y2+30NPj/C5LRAJGAe+TtOQEHvnsLG6dU8p/v7GTe59YScfxLr/LEpEAUcD7KDE+jn/82BT+6rrzeHVTLZ986F2NlReRQaOA95mZcdclY3jks7PZ29jB9f/5Fu/sbPC7LBEJAAV8hLh8UgHP3ncxOWlJ3PHIezy8dBfOqV9eRD44BXwEGZefzrNfuJirzyvk71/YzL0/X0XzES3mLSIfjAI+wqQnJ/Dfn57BN6+dxO8213Ldfy5l3f7DfpclIlFIAR+BzIx7PjSOp/5kHt3djo8/+A4PL92loZQiclYU8BFs5ujhvHj/pcyfWMDfv7CZz/z0PWpbjvpdlohECQV8hMtOTWLBHTP5x49dQOWeJj7y70t4aX2N32WJSBRQwEcBM+O2C0t5/ouXUjI8lXufWMWfLVzN4Y7jfpcmIhFMAR9Fxhek86s/vYgvXTWBF9fX8OF/W8LvNmnCMhE5OQV8lEmMj+P+q8p59gsXk5uWxB8/Vsl9v1hFQ9sxv0sTkQijgI9SU4qzWHTfJXz5wxN4ZWMtV/3wTZ6u3KeLo0SkjwI+iiUlxPFnV5bz4v2XMC4/na89s45bFixjR51WjBIRBXwgjC/I4Ok/mcc//eEFbDnYyh/8x1L++bdbNDulSIxTwAdEXJxx65xSFn/lMm6oKObBN3Zyxb++yaK11eq2EYlRCviAyU1P5gc3V/DLe+eRm57Eny1czacWLGPDgWa/SxORIaaAD6iZo3NYdN8l/MPHprCjro3rH3iLrz69VlfCisQQBXyAxccZn75wNG98bT73XDqWRWuqmf/9N/jhK1tpO6b+eZGgU8DHgMyURL5x7WR+9+XLuHJyAT9avIPL/uV1fvZOFce7evwuT0TCRAEfQ0pzU3ngthk894WLKS9M5zuLNnLFD97gmZX76dZMlSKBo4CPQRUl2Sy8ey6P/tFsslMT+erTa/nIvy/h+XXVmpJYJEAU8DHKzJg/sYDf3HcJD356BgD3/WI1f/AfS3lxfY2CXiQALJLGSM+aNctVVlb6XUZM6u5xPL+umh+9tp2d9e1MKEznC5eP57qpI4mPM7/LE5FTMLOVzrlZJ31OAS/99Qb9A4t3sL2ujTF5adx72Thuml5MUoI+8IlEGgW8nLWeHscrmw7yn4t3sLG6haLMFP740jHcOqeUtOQEv8sTkRAFvHxgzjmWbG/gwTd2sGxXE5kpCdw+dzR3XlRGQWaK3+WJxDwFvAyK1XsPsWDJLn678SAJccaN04r53MVjOG9kpt+licQsBbwMqj2N7Ty8dDfPrNzPkc5u5o3N5XOXjOGKSQU6ISsyxBTwEhbNHZ0sXLGXn71TRU3zUUpyhnHH3NF8alYpWamJfpcnEhMU8BJWXd09vLKplkffruK9qiZSEuO4saKYO+aNZkpxlt/liQSaAl6GzMbqZn6+bA/Prq7mSGc3FSXZfHpOKddVjCA1SaNvRAabLwFvZj8FrgPqnHNTBvIaBXxwNB/p5Fer9vPzZXvYWd9ORnICN00v5pY5JZw/Ukf1IoPFr4D/ENAGPKaAj13OOVZUHeIXy/fw4oaDHO/q4YLiLD41u4TrK0aSNUx99SLnwrcuGjMrA55XwAvA4Y7jPLv6AE+u2MeWg60kJ8RxzZQiPjmzhIvG5RKnETgiZy2iA97M7gHuASgtLZ25Z8+esNUjkcE5x/oDzTxduZ/n1hyg5WgXI7NSuGl6MR+fOYpx+el+lygSNSI64PvTEXzsOdrZzSubavnVqv0s2VZPj4OKUVncNL2Y6ytGkpee7HeJIhFNAS9Roa7lKM+tqebXqw+wqaaF+Djj0vI8bqgYydXnF5GuOXBE/h8FvESdrQdbeXbNARatqebA4SMkJ8Rx1eRCrq8YwfyJBaQkxvtdokhE8GsUzUJgPpAH1ALfcc49crrXKODlRD09jlV7D/Hcmmpe2lBDQ9tx0pMTuGpyAR+dOpJLy/MU9hLTdKGTBEJXdw/LdjXxm7XVvLzpIIc7OvvC/popI5g/MV9hLzFHAS+B09ndwzs7G3lxXU1f2KcmxXP5xAI+MqWIyyfmk5GiMfYSfAp4CbTO7h6W72rixQ01vLLxIA1tx0mKj+Pi8blcfX4RV04uoCBDc9dLMCngJWZ0h/rsX95wkJc3HWRf0xHMYHpJNledV8jV5xUyLj8dM11UJcGggJeY5Jxja20rr2ys5ZVNB9lwoAWA0bmpXDmpkCsnFzC7LEdrzUpUU8CLADXNR3htcx2vbqrl3V2NHO/qIT05gUvL87h8UgHzJ+arK0eijgJe5ATtx7p4e0cDi7fU8frWOmpbjgFwQXEW8yfmM39iPtNKhmuFKol4CniR03DOsammhde31PHG1npW7T1Ej4OsYYlcMj6Pyybk86EJ+RRl6eheIo8CXuQsNHd0snRHPW9urWfJ9vq+o/sJhelcWp7PJeV5XDgmRwuYSERQwIt8QM45thxsZen2epZub2D57iaOd/WQFB/HjNHZXDI+j4vH53FBcRYJ8TpZK0NPAS8ySI52drOiqom3tjewdHsDm2q8kTkZKQnMHZvLxeNyuWh8HuUFGoopQ0MBLxImjW3HeGdnI+/sbOCtHQ3sazoCQF56EnPH5jJvXC7zxuYyJi9NgS9hoYAXGSL7mjp4NxT47+5q7Ou/L8xMZu7YXOaOzeXCMTkKfBk0pwt4nSUSGUQlOamU5KRy8+wSnHPsbmjnnZ2NLN/dxDs7G3luTTUA+RnJzBmTw4VjcpgzJocJBRlaslAGnQJeJEzMjLH56YzNT+f2uaP7An/57iaW7/JC/4V1NYA3JHN22XBmleUwuyyHC4qzdIWtnDMFvMgQ6R/4t84pxTnH/kNHeG93E+/tbmJFVRO/21wHQHJCHBUl2cwaPZzZZTnMKB1OVqpmx5Szoz54kQhS33qMyqomKvccorKqiY3VLXT1eP9HywvSmTl6ODNGD2dG6XDG5asfX3SSVSRqdRzvYu2+Zlbu8UJ/9d7DNB/pBLxuneml2cwo9QK/oiRLc+DHIJ1kFYlSqUkJ3lDLcbmAt4ThroY2Vu05zMo9h1i19xBvbK0HwMw7yp9eMpxppdlUjMpmQmG6LsCKYTqCF4lyzUc6WbPvMGv2HmbNvkOs3neYwx3eUX5qUjxTirOYVuIFfkVJFsXZw9S1EyA6ghcJsKxhiVw2IZ/LJuQD3vQKexo7vNAP3R59u4rj3T0A5KYlMXVUFlNDgT91VDZ56cl+NkHCRAEvEjBmRlleGmV5adw0vRiA4109bK5pYd3+w6zd38zafYd5Y1s9vR/gR2alcEEo9KcUZ3FBcRY5aUk+tkIGgwJeJAYkhYZdVpRkc0fosfZjXWys9kJ//YFm1u9v5uWNtX2vKc4expTiTKaMzGJKcRbnF2dqQZQoo4AXiVFpyQnMCV1J26v5SCcbq5vZcKCZ9Qda2HDg/aFfkJHshf3IzNAti1HD1acfqRTwItIna1giF43L46JxeX2PtR7tZFN1C+sPNLOpuoWN1S28ua2e7tD4/MyUBCaP8ML+vJGZTB6RQXlBhq7EjQAKeBE5rYyURC4cm8uFY3P7Hjva2c2Wg61srG5mY3ULm2taWPjeXo50dgOQGG+My0/nvBGZTO67ZZCrk7lDSgEvImctJTGeaSXZTCvJ7nusu8eba2dzTQubalrYVN3CWzsa+NXqA33b5Gcke2FflMGkERlMLMxkfEG6jvbDRAEvIoMiPs4YX5DO+IJ0rq8Y2fd4Y9sxNte0suVgC5trWtlc08L/7GzsG7aZEGeMzU9jYlEmk4oymFiYwcSiDIqzh2mGzXOkgBeRsMpNT+aS8mQuKf99v35ndw9VDe1sPtjK1oMtbD3Yyuq9h/jN2uq+bdKS4ikv9AJ/QlEGEwrTmVCYQUFGsk7qDpACXkSGXGJ8HOWFGZQXZkC/o/3Wo51sq21jayj4t9W28ermWp6q3Ne3TdawRCYUplNemMGEAi/0ywszyEtPUvCfQAEvIhEjIyWRmaOHM3P08Pc93tB2jG21rWw72MrW2ja217by/NpqWo529W2TnZpIeUE64wsyKC9Ip7zQ6y4qykyJ2eBXwItIxMtLTyYvPfl9wzedc9S1HmN7bRvb61rZVtvGjrpWXtpQw8LQXDwA6ckJjMtPY1zo/MD4fO9raU5q4CdiU8CLSFQyMwozUyjMTHlf/75zjoa24+yoa2NHfRs7alvZUd/G2zsa+NWq34/oSYw3ynLTGJefzrgC76u3IEsamQGZdlkBLyKBYmbkZySTn5HcN81yr5ajneyqb/fCv66NnfVtbKtr5dXNtX0XboE3nHNcfpoX+Hm94Z9GcfawqDrqV8CLSMzITEn8f+P3wZuMbW9TBzvr29hV3x762saL62v6pl4G76h/dG4aY/LSGJvnfR2Tl8aY/DTy0yNvdE9YA97MrgH+A4gHHnbOfS+c+xMR+SCSEuL6xvCfqKn9OLt6g7+hjaqGdnbVt/Pm1vq+sfzg9fWX5aVSluuFf1m/PwDZqf7MzBm2gDezeOC/gA8D+4EVZrbIObcpXPsUERlsOWlJ5KTlMKss532Pd/c4qg8fYVdDO7vr26hq7GBXQztr9x/mxfU19OvxIWtYohf4uamMzk3r+0NQlptGdmpi2I78w3kEPwfY4ZzbBWBmTwI3Agp4EYl68XFGSU4qJTmpfYut9DrW1c2+piNUNbRT1djO7tDXFVWHeG5tNf0X0stMSWBiUQb/+yfzBj3owxnwxcC+ft/vBy48cSMzuwe4B6C0tDSM5YiIDI3khPhTdvl44d/BnsYOqho72NPYzvGunrAcxft+ktU5twBYAN6arD6XIyISVl74ZzC+ICPs+wrneJ8DQEm/70eFHhMRkSEQzoBfAZSb2RgzSwJuARaFcX8iItJP2LponHNdZnYf8DLeMMmfOuc2hmt/IiLyfmHtg3fOvQi8GM59iIjIyUXPNbciInJWFPAiIgGlgBcRCSgFvIhIQJlzkXNtkZnVA3vO4iV5QEOYyolUsdhmiM12x2KbITbbfS5tHu2cyz/ZExEV8GfLzCqdc7P8rmMoxWKbITbbHYtththsd7jarC4aEZGAUsCLiARUtAf8Ar8L8EEsthlis92x2GaIzXaHpc1R3QcvIiKnFu1H8CIicgoKeBGRgIr4gDeza8xsq5ntMLOvn+T5ZDN7KvT8cjMr86HMQTeAdn/ZzDaZ2Toze83MRvtR52A6U5v7bfdxM3NmFoihdANpt5ndHPp9bzSzXwx1jYNtAP++S83sdTNbHfo3fq0fdQ4mM/upmdWZ2YZTPG9m9qPQz2Sdmc0455065yL2hjfN8E5gLJAErAXOO2GbPwUeCt2/BXjK77qHqN2XA6mh+/dGe7sH0ubQdhnAEmAZMMvvuofod10OrAaGh74v8LvuIWjzAuDe0P3zgCq/6x6Edn8ImAFsOMXz1wIvAQbMBZaf6z4j/Qi+b+Fu59xxoHfh7v5uBH4Wuv8McKWFa4nyoXPGdjvnXnfOdYS+XYa3YlY0G8jvGuDvgH8Gjg5lcWE0kHbfDfyXc+4QgHOubohrHGwDabMDMkP3s4DqIawvLJxzS4Cm02xyI/CY8ywDss1sxLnsM9ID/mQLdxefahvnXBfQDOQOSXXhM5B293cX3l/+aHbGNoc+spY4514YysLCbCC/6wnABDN728yWmdk1Q1ZdeAykzX8D3G5m+/HWlPji0JTmq7P9f39Gvi+6LefGzG4HZgGX+V1LOJlZHPBD4E6fS/FDAl43zXy8T2pLzOwC59xhP4sKs1uBR51zPzCzecDjZjbFOdfjd2HRJNKP4AeycHffNmaWgPdxrnFIqgufAS1YbmZXAd8CbnDOHRui2sLlTG3OAKYAb5hZFV4f5aIAnGgdyO96P7DIOdfpnNsNbMML/Gg1kDbfBfwvgHPuXSAFb0KuIBvQ//uzEekBP5CFuxcBnw3d/wSw2IXOWESxM7bbzKYDP8YL92jvk4UztNk51+ycy3POlTnnyvDOO9zgnKv0p9xBM5B/48/iHb1jZnl4XTa7hrDGwTaQNu8FrgQws8l4AV8/pFUOvUXAZ0KjaeYCzc65mnN5w4juonGnWLjbzP4WqHTOLQIewfv4tgPvBMYt/lU8OAbY7u8D6cDToXPKe51zN/hW9DkaYJsDZ4Dtfhm42sw2Ad3A15xzUfspdYBt/grwEzP7Et4J1zuj/cDNzBbi/aHOC51b+A6QCOCcewjvXMO1wA6gA/ijc95nlP/MRETkFCK9i0ZERD4gBbyISEAp4EVEAkoBLyISUAp4EZGAUsBLVDCz+Wb2fOj+DWeYbTLbzP70A+zjb8zsq+dS52C8r5mVnWbGwYfN7LzQ/arQuHjM7J1+r71tMOqW6KeAF1+ZWfzZvsY5t8g5973TbJKNN8vokAldRR12zrk/ds5tOsnjF4XulgEKeAEU8BImoSPJLWb2hJltNrNnzCw19FyVmf2zma0CPmlmV5vZu2a2ysyeNrP00HbXhN5jFfCH/d77TjN7IHS/0Mx+bWZrQ7eLgO8B48xsjZl9P7Td18xsRWie7e/2e69vmdk2M3sLmHiKtjxqZg+ZWWVo2+v61bHIzBYDr5lZjpk9G9rHMjOb2u9tKkJt3G5md4den27eXP6rzGy9mfWfUTHhFD+7N042PYOZtYXufg+4NNT2L5nZEjOb1m+7t8ysYgC/QgkABbyE00Tgv51zk4EW3n9U3eicmwH8Dvg2cFXo+0rgy2aWAvwEuB6YCRSdYh8/At50zlXgzbW9Efg6sNM5N8059zUzuxpv7pY5wDRgppl9yMxm4l35PA3vCsLZp2lLWej1HwUeCtVHaJ+fcM5dBnwXWO2cmwp8E3is3+unAlcA84C/NrOReFMefyzU7suBH5j1TXV9up/d6XwdWBpq+7/hXel9J4CZTQBSnHNrB/heEuUU8BJO+5xzb4fu/xy4pN9zT4W+zsVb0OFtM1uDN6/QaGASsNs5tz10ifrPT7GPK4AHAZxz3c655pNsc3XothpYFXrvcuBS4NfOuQ7nXAv/fz6U/v7XOdfjnNuONw/MpNDjrzrneuf4vgR4PFTLYiDXzHrnNH/OOXfEOdcAvI73x8KAfzSzdXh/6IqBwtD2p/vZnY2ngevMLBH4HPDoB3wfiUIRPReNRL0T58Ho/3176KvhheSt/Tfs360wCAz4J+fcj0/Yx5+fxXucqi3tJ254Fq//NJAPzHTOdZo3S2bKabY/a865DjN7FW8xiZvxPg1JjNARvIRTqXlzeYN34u+tk2yzDLjYzMYDmFlaqCthC1BmZuNC2916ktcCvIa3ZCFmFm9mWUAr3vTCvV4GPtevb7/YzArwlv67ycyGmVkGXnfQqXzSzOJC9YwFtp5km6V4oY2ZzQcaQp8MAG40sxQzy8WbcGoF3tTWdaFwvxzvk0uvgfzsTubEtgM8jNeVtaJ3VSiJDQp4CaetwBfMbDMwnFBXSn/OuXq8PuKFoa6Kd4FJzrmjwD3AC6GTrKeaEvl+4HIzWw+sxFvbsxGvy2eDmX3fOfcK8Avg3dB2zwAZzrlVeF1Fa/FWxFpxmrbsBd4Lbff5UH0n+hu8/v11eCc7P9vvuXV4XTPLgL9zzlUDTwCzQjV9Bu+PWq8z/uxOYR3QHTrh/CUA59xKvH78/xnge0hAaDZJCQszKwOed85N8buWc2Vmj+K15Rm/a/kgQid038D7w6kVkWKIjuBFAszMPgMsB76lcI89OoIXEQkoHcGLiASUAl5EJKAU8CIiAaWAFxEJKAW8iEhA/R86oO3lFJOm3AAAAABJRU5ErkJggg==\n",
|
||
"text/plain": [
|
||
"<Figure size 432x288 with 1 Axes>"
|
||
]
|
||
},
|
||
"metadata": {
|
||
"needs_background": "light"
|
||
},
|
||
"output_type": "display_data"
|
||
}
|
||
],
|
||
"source": [
|
||
"import matplotlib.pyplot as plt\n",
|
||
"import numpy as np\n",
|
||
"\n",
|
||
"\n",
|
||
"x = np.linspace(0, 1, 100)\n",
|
||
"y = -np.log(x) # 二分类化简\n",
|
||
"\n",
|
||
"plt.plot(x, y)\n",
|
||
"plt.xlabel('predicted probability')\n",
|
||
"plt.ylabel('loss')"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"可以看出，该函数是 __凸函数__ ，求导时能够得到全局最优值。"
|
||
]
|
||
}
|
||
],
|
||
"metadata": {
|
||
"kernelspec": {
|
||
"display_name": "Python 3",
|
||
"language": "python",
|
||
"name": "python3"
|
||
},
|
||
"language_info": {
|
||
"codemirror_mode": {
|
||
"name": "ipython",
|
||
"version": 3
|
||
},
|
||
"file_extension": ".py",
|
||
"mimetype": "text/x-python",
|
||
"name": "python",
|
||
"nbconvert_exporter": "python",
|
||
"pygments_lexer": "ipython3",
|
||
"version": "3.8.4"
|
||
}
|
||
},
|
||
"nbformat": 4,
|
||
"nbformat_minor": 4
|
||
}
|