ãã¯ãããŠã®ãã¿ãŒã³èªèãã®ç¬¬10ç« ãã¯ã©ã¹ã¿ãªã³ã°ãã®è§£èª¬ã§ãã
åå(10.3ãŸã§)ã¯ãéšåç©ºéæ³ãšå€ããé£ããæ°åŒãåºãŠããçè§£ããããã£ãã§ãã10.4 確çã¢ãã«ã«ããã¯ã©ã¹ã¿ãªã³ã°ã§æ°åŒãåºãŠããŠé£ãããªããã€ãã«EMã¢ã«ãŽãªãºã ã¯çè§£ãæŸæ£ããŸãã
ãšãã£ãŠããçè§£ã§ããªãã£ããšãããããæéãããããã«ã䜿ããã®èªä¿¡ããªãã£ãã®ã§ã»ãšãã©èªãŸãé£ã°ããŸãããã¿ã€ãã«ã®ãå
šåãã«æ¥ããè¡çºã§ãããèš±ããã ããã
â»ã¯ããã¿å匷系ã¯èšäºãæç³»ç€ŸäŒäººãã¯ããã¿ã§æ©æ¢°åŠç¿ãæ°åŒå«ããŠçè§£ãã奮éèšãã«ãŸãšããŸããã
åèãªã³ã¯
- ã¯ããã¿ 10ç« ã¯ã©ã¹ã¿ãªã³ã° åå(Slideshare)
- ã¯ããããŠã®ããã¿ãŒã³èªè茪èªäŒ 10ç« åŸå(Slideshare)
- ã¯ãããŠã®ãã¿ãŒã³èªè 10ç« : æ°åŒåè
å 容
10.1 é¡äŒŒåºŠãšéé¡äŒŒåºŠ
10.1.1 è·é¢ã®å ¬ç
ã¯ã©ã¹ã¿ãªã³ã°ã¯ãããŒã¿ãã¯ã©ã¹ã¿éã®é¡äŒŒåºŠããŸãã¯éé¡äŒŒåºŠã«åŸã£ãŠã䌌ããã®ãéããŠã¯ã©ã¹ã¿ãäœãæ¹æ³ã§ããç»åäžã®è²ãæãããªã©ã®é¡äŒŒããæ§è³ªãæã€é åãä»é åãšåºå¥ããã»ã°ã¡ã³ããŒã·ã§ã³ããã¯ãã«éååãªã©åºãåéã§å©çšãããŠããŸãã
ããŒã¿ãã¯ã©ã¹ã¿éã®é¡äŒŒåºŠã枬ãåºæ¬çãªå°ºåºŠãè·é¢ã§ãããã尺床d(x,y)ãè·é¢ã§ããããã«ã次ã®è·é¢ã®å
¬çãæºããå¿
èŠããããŸãã
- éè² æ§: d(x,y) â§ 0
- åå°åŸ: d(x,y) = 0 ãšãªãã®ã¯ x = y ã§ãããšãã®ã¿ã«éã
- 察称æ§: d(x,y) = d(y,x)
- äžè§äžçåŒ: d(x,z) ⊠d(x,y) + d(y,z)
10.1.2 ãã³ã³ãã¹ããŒè·é¢
è·é¢å°ºåºŠã«ã¯ããã䜿ããããŠãŒã¯ãªããè·é¢ããããŸãããã以å€ã«ããã³ã³ãã¹ããŒè·é¢ãšåŒã°ããäžè¬çãªè·é¢ã®å®çŸ©ãããæ§ã
ãªè·é¢ã掟çãããããšãã§ããŸãã
d次å
ã®ããŒã¿ãNåãããiçªç®ã®ãã¯ãã«ã$x_i=(x_{i1},\ldots,x_{id})^T$ãšãããšã2ã€ã®ãã¯ãã«$x_i, x_j$éã®è·é¢ã¯ä»¥äžã®åŒã§ãã
d(x_,x_j)=( \sum_{k = 1}^d |x_{ik} - x_{jk}|^a )^{1/b}
ãã®aãšbã®å€ãå€ããŠæ§ã
ãªè·é¢ãçã¿åºããŸãã
aã®å€ã倧ãããã°ç¹åŸŽéã®å·®ã«å€§ããªéã¿ãå ããããŸãã
bã®å€ã倧ãããã°å·®å环ä¹åã«å¯Ÿããéã¿ãå°ãããªããŸãã
以äžãaãšbã®å€ãå€ããŠæŽŸçããè·é¢ã§ãã
â»äžè¬çã«ãã³ã³ãã¹ããŒè·é¢ã¯a=bã§å®çŸ©ãããŠããããšãå€ãããã§ãããŸããa!=bã ãšè·é¢ã®å
¬çãæºãããªããªãããšããããæŽŸçãããå Žåã«ã¯æ³šæãå¿
èŠã§ãã詳ããã¯ã³ã¡ã³ãæ¬ãåç
§ãã ããã
| No. | a | b | è·é¢ | åŒ |
|---|---|---|---|---|
| 1 | 1 | 1 | ãã³ã³ãã¹ããŒè·é¢ (ãã³ããã¿ã³è·é¢/åžè¡å°è·é¢) |
$d(x_i, x_j) = \sum_{k = 1}^d |x_{ik} - x_{jk}|$ |
| 2 | 2 | 2 | ãŠãŒã¯ãªããè·é¢ | $d(x_i, x_j) = (\sum_{k = 1}^d |x_{ik} - x_{jk}|^2)^{\frac{1}{2}}$ |
| 3 | 2 | 1 | ãŠãŒã¯ãªããè·é¢ã®2ä¹ | $d(x_i, x_j) = \sum_{k = 1}^d |x_{ik} - x_{jk}|^2$ |
| 4 | â | â | ãã§ãã·ãè·é¢ | $d(x_i, x_j) = \lim_{a \rightarrow \infty}(\sum_{k = 1}^d |x_{ik} - x_{jk}|^a)^{\frac{1}{a}}$ $= \max_k |x_{ik} - x_{jk}|$ |
ãã³ã³ãã¹ããŒè·é¢ã®æŽŸçã§ã¯ãªãã§ãããè·é¢ã®å ¬çãæºãããã®ãšããŠãã£ã³ãã©å°ºåºŠããããŸããããŒã¿æ£èŠåã®ä»çµã¿ãå å ãããŠããŸããåç¹ã«è¿ãã»ã©åæ¯ãå°ãããªããéã¿ã¥ããããŸãã
d(x_i, x_j) = \sum_{k = 1}^d \frac{|x_{ik} - x_{jk}|}{|x_{ik}| + |x_{jk}|}
ãã¯ãã«éã®è§åºŠ($\cos \theta$)ãçšããã³ãµã€ã³é¡äŒŒåºŠããããŸããèšäºãèšèªåŠç100æ¬ããã¯-87:åèªã®é¡äŒŒåºŠãã§ãããŸããã
â»æ¹åäœåŒŠã¯ãã³ãµã€ã³é¡äŒŒåºŠã®ããšããæ¹åäœåŒŠãã¯ãã¯ãã«ã ãšæã£ãŠããã®ã§ãæ··ä¹±ããŸããã2æ¬ã®ãã¯ãã«ãã©ããããåãåããåããŠããã®ããè¡šãææšã§ã1ã«è¿ãã»ã©2æ¬ã®ãã¯ãã«ã¯åãåãã«è¿ãã§ããåãåŸãå€ã¯-1ãã1ãŸã§ã§ãããè² ã®å€ãããã®ã§è·é¢ã®å
¬çã¯æºãããŠããŸããã
d(x_i, x_j) = \frac{x_i x_j}{|x_i| |x_j|}
= \frac{\sum_{k = 1}^d x_{ik} x_{jk}}{\sqrt{(\sum_{k = 1}^d x_{ik}^2) (\sum_{k = 1}^d x_{jk}^2) }}
è·é¢ã®çš®é¡ã¯ãã®èŸºãéåžžã«ããããããã£ãã§ãã
- ã¯ã©ã¹ã¿ãŒåæã®ææ³â ïŒæŠèŠïŒ
- ãã®ããŒã¿ã»ããã«ã¯ã©ã®è·é¢ãçšããã°ããã®ïŒ
10.2 ééå±€ã¯ã©ã¹ã¿ãªã³ã°(K-平忳)
éæ¹è£
ã¯ã©ã¹ã¿ãªã³ã°ã®ä»£è¡šçææ³ã§ãã**K-平忳(K-meansæ³)**ã§ããd次å
ã®Nåã®ããŒã¿$D=\{x_1, \ldots, x_N\}, x_i \in \mathbb R^d$ãããŒã¿éã®é¡äŒŒæ§(è·é¢)ã尺床ã«ãããããå®ããKåã®ã¯ã©ã¹ã¿ã«åé¡ããŸããèšäºãK-means æ³ã D3.js ã§ããžã¥ã¢ã©ã€ãºããŠã¿ããã¯èŠãã ãã§æŠèŠãçè§£ã§ããã»ã©ããã°ãããèšäºã§ãã
å¥èšäºãèšèªåŠç100æ¬ããã¯-97(scikit-learn䜿çš):k-meansã¯ã©ã¹ã¿ãªã³ã°ãã§ã䜿ããŸããã
- åã¯ã©ã¹ã¿ã®ä»£è¡šãã¯ãã«ã®éå: $M = \{ \mu_i, \ldots, \mu_K \}$
- kçªç®ã®ä»£è¡šãã¯ãã«ãæ¯é ããã¯ã©ã¹ã¿: $M(\mu_k)$
- içªç®ã®ããŒã¿ãkçªç®ã®ä»£è¡šãã¯ãã«ãæ¯é ããã¯ã©ã¹ã¿$M(\mu_k)$ãžã®åž°å±æç¡ã衚ãåž°å±å€æ°: $q_{ik}$(äžåŒåç §)
{q_{ik} = \left\{
\begin{array}{ll}
1 & \boldsymbol{x_i} \in M(\boldsymbol{\mu_k})\\
0 & otherwize
\end{array}
\right.}
K平忳ã®è©äŸ¡é¢æ°ã以äžã«èšå®ã$q_{ik}$ã1ã®å Žåã«ä»£è¡šãã¯ãã«ãšã®è·é¢ãæå°åãããŸãã
J(q_{ik}, \mu_k)= \sum_{i=1}^{N}\sum_{k=1}^{K}q_{ik}||x_i-\mu_k||^2
$\mu_{ik}$ãå埮åã«ããæé©åããŸãã
\frac{\partial J(q_{ik}, \mu_k)}{\partial \mu_k}=2\sum_{i=1}^Nq{ik}(x_i-\mu_k)=0\\
â\mu_k=\frac{\sum_{i=1}^Nq_{ik} x_i}{\sum_{i=1}^N q_{ik}
}
æåŸã®åœ¢ãèŠããšãåã¯ã©ã¹ã¿ã®ä»£è¡šãã¯ãã«ã¯ããã®ã¯ã©ã¹ã¿ã«åž°å±ããããŒã¿ã®å¹³åãã¯ãã«ã«ãªã£ãŠããããšãããããŸãã$\mu_{ik}$ãæé©åããŸãããã$q_{ik}$ãšåæã«æé©åãé£ããã®ã§ãæ¬¡ã®æé ã§é次çã«æé©åã宿œããŸãã
0. åæå
Nåã®ããŒã¿ãã©ã³ãã ã«Kåã®ã¯ã©ã¹ã¿ã«æ¯ãåããããããã®ã¯ã©ã¹ã¿ã®å¹³åãã¯ãã«ãæ±ãã$\mu_k(k=1, \ldots, K)$ãšããŸãã
1. qæé©å
$\mu_k$ãåºå®ããåž°å±å€æ°$q_{ik}$ã以äžã«åŸã£ãŠæ±ºå®ã
q_{ik} = \left\{
\begin{array}{ll}
1 & k = argmin_j ||\boldsymbol{x_i}-\boldsymbol{\mu_j}||^2\\
0 & otherwize
\end{array}
\right.
K-平忳ã®åæå
ã¯åæå€ã«äŸåãããããæé©è§£ã«è¿ãè§£ãåŸãããã«ã¯äœåãåæå€ãå€ããŠå®è¡ããå¿
èŠããããŸããå代衚ãã¯ãã«ãæ¯é
ããé åã¯ã代衚ãã¯ãã«éã®ãããã€å¢çã§æ±ºãŸããŸãã
代衚ãã¯ãã«ã¯å¹³åãã¯ãã«ãªã®ã§ãããŒã¿ãã¯ãã«ãšäžè¬ã«äžèŽãããŸããã代衚ãã¯ãã«ãããŒã¿ãã¯ãã«ã«éå®ããæ¹æ³ã**K-ã¡ãã€ãæ³(K-medoidsæ³)**ãšèšããŸãã代衚ãã¯ãã«ã«å¹³åãã¯ãã«ã䜿ããããK-平忳ã¯ããŒã¿éã®è·é¢ãå®çŸ©ããå¿
èŠããããŸããäžæ¹ã§ã代衚ãã¯ãã«ã«ããŒã¿ãã¯ãã«ã䜿ãK-ã¡ãã€ãæ³ã¯ããŒã¿éã®éé¡äŒŒåºŠè¡å(è·é¢ã®è¡åã§ãOK)ãå®çŸ©ã§ããã°èšç®ã§ããŸãã
ã¡ãã€ãã¯ãã¯ã©ã¹ã¿å
ã®ããŒã¿ç¹ã®ãã¡ããã®ç¹ä»¥å€ã®ã¯ã©ã¹ã¿å
ã®ä»ã®ç¹ãšã®æ¯é¡äŒŒåºŠ(è·é¢ã§ãOK)ã®ç·åãæåãã«ãªãç¹ã§ãã以äžãæºãã$x_i$ã代衚ãã¯ãã«$\mu_k$ã®é åã®ã¡ãã€ãã§ãã
i = \arg \min_{x_i \in M(\mu_k)}\sum_{y \in \{M(\mu_k) - x_i\}} d(x_j, y)
K-平忳ãè·é¢ã®2ä¹ã§èª€å·®ãè©äŸ¡ãããã®ã«å¯ŸããŠãK-ã¡ãã€ãæ³ã¯1ä¹ãªã®ã§ãå€ãå€ã®åœ±é¿ãå°ãªãã§ãã
2. ÎŒã®æé©å
$q_{ik}$ãåºå®ããã»ã³ããã€ã$\mu_k$ãæ±ºãã(äžã§ç޹ä»ããå埮åã®åŒ)ã
3. ç¹°ãè¿ã
ã1. qæé©åããšã2. ÎŒã®æé©åããåæãããŸã§ç¹°ãè¿ããŸãã
10.3 éå±€ã¯ã©ã¹ã¿ãªã³ã°(èåæ³)
ã¯ã©ã¹ã¿ãªã³ã°ãããŠããªãããŒã¿ãããé¡äŒŒåºŠãé«ãé ã«èåããŠåŸã
ã«å€§ããªã¯ã©ã¹ã¿ãäœããæçµçã«1ã€ã®ã¯ã©ã¹ã¿ã«çµ±åããŠããæ¹æ³ãéå±€çã¯ã©ã¹ã¿ãªã³ã°ãŸãã¯èåæ³ãšèšããŸããããŒã¿ã»ã¯ã©ã¹ã¿ãèåããéçšã**æš¹ç¶å³(dendrogram)**ãšåŒã°ãã圢ã§è¡šçŸã§ããŸãã
以äžãèåæ³ã®æé ã§ãã
- n = NãšããŸã
- n à n ã®è·é¢è¡åãäœããŸã
- æãè·é¢ãè¿ã2ã€ã®ããŒã¿ãã¯ã©ã¹ã¿ããŸãšããŠãäžã€ã®ã¯ã©ã¹ã¿ã«ããŸã
- n = n - 1ãšããŸã
- n > 1 ã§ããã°2ãžæ»ããŸããn = 1 ã§ããã°çµäºããŸãã
é¡äŒŒåºŠã枬ã尺床ã¯ããã³ã³ãã¹ããŒè·é¢ããã£ã³ãã©å°ºåºŠãªã©ã䜿ããŸããã¯ã©ã¹ã¿éã®é¡äŒŒåºŠã¯ã以äžã®ãããªæ§ã ãªãã®ããããŸãã
åè: éå±€çã¯ã©ã¹ã¿ãªã³ã°å ¥éã®å ¥é
10.3.1 åé£çµæ³
2ã€ã®ã¯ã©ã¹ã¿éã§æãé¡äŒŒåºŠã®é«ãããŒã¿éã®è·é¢ãã¯ã©ã¹ã¿éã®è·é¢ã«ããæ¹æ³ã**åé£çµæ³(single linkage method)**ãšèšããŸã(æçè·é¢æ³ãšãåŒã¶ããã§ã)ã
以äžãåé£çµæ³ã«ããã¯ã©ã¹ã¿ãªã³ã°ã®ç¹åŸŽã§ãã
- ã¯ã©ã¹ã¿ã«1ã€ããŒã¿ã远å ããããšãä»ã¯ã©ã¹ã¿ãšã®è·é¢ã¯å°ãããªããããŸãã¯å€åããªã(æçè·é¢ãªã®ã§åœç¶)ã
- 倧ããªã¯ã©ã¹ã¿ãã§ããåŸåããããŸãã
- ããã¯ã©ã¹ã¿ããåãè·é¢ã«2ã€ã®ã¯ã©ã¹ã¿ãããå Žåãã©ã¡ããéžãã§ãçµæã¯åã(éäžçµéã¯å°ãå€ãããã2ã€ãæçè·é¢ãªã®ã§æ¬¡ã®èåã§2ã€ç®ã®ã¯ã©ã¹ã¿ã察象ãšãªrimasu
- )ã
- è¿ãããŒã¿ãå¥ãªã¯ã©ã¹ã¿ã«å±ããé£é广ã衚ããå ŽåããããŸã(è¿ãããŒã¿ããèåããŠããã®ã§èã®ããã«é·çްããªã£ãŠããã€ã¡ãŒãž)
10.3.2 è¶ è·é¢
2ã€ã®ããŒã¿$x_i, x_j$ãèåããçŽåã®ã¯ã©ã¹ã¿éã®è·é¢ã**è¶
è·é¢(ultrametric)**ãšèšãã$\widetilde{d}(x_i,x_j)$ã§è¡šããŸãã
â»ãè¶
è·é¢ãã¯ãè¶
è·é¢æ³ã®ãããªãã®ãããã®ããšæ··ä¹±ããŸããããè·é¢ã®çš®é¡ã»åŒã³æ¹ã«ã€ããŠã®è§£èª¬ã®ç« ã§ãããããã«ããã»ã»ã»
10.3.3 å®å šé£çµæ³
åé£çµæ³ãæè¿åè·é¢ãåºæºã«ããã®ã«å¯ŸããŠãå®å šé£çµæ³(complete linkage method)ã¯æé é£è·é¢ãã¯ã©ã¹ã¿éã®è·é¢ã«ããæ¹æ³ã§ãã以äžãç¹åŸŽã§ãã
- ã¯ã©ã¹ã¿ã«1ã€ããŒã¿ã远å ãããšãä»ã¯ã©ã¹ã¿ãšã®è·é¢ã¯å€§ãããªã(åé£çµæ³ã®é)ãããŸãã¯å€æããªãã
- 倧ããªã¯ã©ã¹ã¿ã«ãªãã«ãããåããããªãµã€ãºã®ã¯ã©ã¹ã¿ãã§ããåŸåã
- é£é广ã¯çŸããªã(é ãããŒã¿ãéžãã§ãããã)
10.3.4 çŸ€å¹³åæ³
çŸ€å¹³åæ³(group average method)ã¯2ã€ã®ã¯ã©ã¹ã¿å ã®å šãŠã®ããŒã¿å¯Ÿéã®è·é¢ã®å¹³åã§ã¯ã©ã¹ã¿éã®è·é¢ã決ããŸããã¯ã©ã¹ã¿å ããŒã¿ã®å¹³åãã¯ãã«éã§ã¯ãªãã®ã泚æç¹ã§ãã
10.3.5 ãŠã©ãŒãæ³
ãŠã©ãŒãæ³(Ward method)ã¯ãã¯ã©ã¹ã¿AãšBã®è·é¢ããããããèåãããšãã®ã¯ã©ã¹ã¿å å€åã®å¢åã§å®çŸ©ããè·é¢ã®å°ããªã¯ã©ã¹ã¿ããèåããŸããã¯ã©ã¹ã¿å å€åã®å¢åã¯ä»¥äžã®åŒã§è¡šããŸãã
\begin{align}
d(A, B) & = \sum_{x \in A, B} d(x, \mu_{AB})^2 - (\sum_{x \in A} d(x, \mu_A)^2 + \sum_{x \in B} d(x, \mu_B)^2) \\
& = S_{AB} - (S_A + S_B)
\end{align}
äžèšã®åŒã®è£è¶³ã§ããã€ãŸããå¹³åãšã®è·é¢ã®å·®(åã¯ã©ã¹ã¿å ã®ã°ãã€ããèæ ®)ããã£ãŠãèåã®åºæºãšããŸãã
| åŒ | 説æ |
|---|---|
| $d(x,y)$ | ãŠãŒã¯ãªããè·é¢ |
| $\mu_{AB}$ | ã¯ã©ã¹ã¿AãšBãèåããã¯ã©ã¹ã¿ã®å¹³åãã¯ãã« |
| $\mu_A$ | ã¯ã©ã¹ã¿Aã®å¹³åãã¯ãã« |
| $\mu_B$ | ã¯ã©ã¹ã¿Bã®å¹³åãã¯ãã« |
| S | å€åãã€ãŸãå¹³åããã®è·é¢(åå·®)ã®2ä¹ |
ãŠã©ãŒãæ³ã¯éå±€æ³ã®äžã§ãã£ãšã粟床ãé«ããããã§ãã
ãã以å€ã«éå¿æ³ãã¡ãã£ã¢ã³æ³ããããŸãã
10.4 確çã¢ãã«ã«ããã¯ã©ã¹ã¿ãªã³ã°
K-平忳ãèåæ³ã«ããã¯ã©ã¹ã¿ãªã³ã°ã¯ã1ã€ã®ããŒã¿ã¯1ã€ã®ã¯ã©ã¹ã¿ã«ã®ã¿åé¡ããããããããŒãã¯ã©ã¹ã¿ãªã³ã°ãšåŒã°ããŸããããŒãã¯ã©ã¹ã¿ãªã³ã°ã«å¯ŸããŠã確çã¢ãã«ã䜿ãæå±ã¯ã©ã¹ã¿ã確ççã«æ±ºãããœããã¯ã©ã¹ã¿ãªã³ã°ããããŸããå€ãã®ç¢ºçã¢ãã«ã¯åå³°æ§ã®ç¢ºçååžãã衚çŸã§ããªãã®ã§ãè€æ°ã®ç¢ºçã¢ãã«ã®éã¿ä»ãç·åœ¢åã§å
šäœã®ç¢ºçååžãã¢ãã«åããŸããã¯ã©ã¹ã¿æ°ãKãkçªç®ã®ã¯ã©ã¹ã¿ã®ç¢ºçã¢ãã«ã$p_k(x)$ãšããå
šäœã®ç¢ºçååžã$p(x) = \sum_{k = 1}^K \pi_k p_k(x)$ã§è¡šããŸãã
$\pi_k$ãkçªç®ã®ç¢ºçã¢ãã«ã®éã¿(æ··åæ¯)ã§ãããã®ãããªç¢ºçã¢ãã«ãæ··åååžã¢ãã«ãšèšããŸãã確çã¢ãã«ã«æ£èŠååžã䜿ãå Žåã**æ··åæ£èŠååžã¢ãã«(æ··åã¬ãŠã¹ååžã¢ãã«)**ãšåŒã°ããŸããæ··åæ£èŠååžã¢ãã«ã䜿ãå Žåãæåã«ã¯ã©ã¹ã¿æ°Kãæ±ºããå¿
èŠããããŸãã
â»ãã¯ããããŠã®ããã¿ãŒã³èªè茪èªäŒ 10ç« åŸåã(Slideshare)ãéåžžã«åèã«ãªããèŠãªããçè§£ããŠãããŸããã
10.4.1 æ··åæ£èŠååžã¢ãã«
kçªç®ã®ã¯ã©ã¹ã¿ã衚ãd次å
æ£èŠååžé¢æ°ã以äžã§è©äŸ¡ããŸããèŠã¯å€å€éæ£èŠååžã¢ãã«ã§ããåèã«1倿°ã®æ£èŠååžã2è¡ç®ã«èšèŒã
â»N ã¯ãæ£èŠååžãã衚ãè±èª "normal distribution" ã®é æåããåãããŠãã
\begin{eqnarray}
\mathcal{N}(x|\mu_k, \Sigma_k) & = &
\frac{1}{(2\pi)^{1/d} |\Sigma|^{1/2}} \exp (- \frac{1}{2} (x - \mu_k)^T \Sigma_k^{-1} (x - \mu_k) )\\
\mathcal{N}(x|\mu, \sigma^2) & = &
\frac{1}{(2\pi)^{1/2} |\sigma|^{1/2}} \exp (- \frac{1}{2} (x - \mu)^2 |\sigma|^{1/2} )
\ \ *1倿°æ£èŠååž\\
\end{eqnarray}
å šäœã®ååžã¯ãç·åœ¢åãªã®ã§ã以äžã§è¡šããŸãã
p(x) = \sum_{k = 1}^K \pi_k \mathcal{N}(x|\mu_k, \Sigma), \;\;
0 \leq \pi_k \leq 1, \;\;
\sum_{k = 1}^K \pi_k = 1
$\pi_k$ãæ··åæ¯ãšåŒã³ãæ··åæ£èŠååžã¢ãã«ã®ãã©ã¡ãŒã¿ã¯ä»¥äžã§è¡šããŸãã
\pi = (\pi_1, \ldots , \pi_K), \;
\mu = (\mu_1, \ldots , \mu_K), \;
\Sigma = (\Sigma_1, \ldots , \Sigma)
$\mu_k$ã¯d次å
ãã¯ãã«ã$\Sigma_k$ã¯dÃd察è§è¡åã§ (d+1)d/2åã®ç°ãªãèŠçŽ ãæã€ã®ã§ãå
šäœã§ K(æ··åæ¯)+dK(å¹³åãã¯ãã«)+(d+1)dK/2(å
±åæ£è¡åã®å¯Ÿè§å)åã®ãã©ã¡ãŒã¿ãæ±ããå¿
èŠããããŸãã
â»æ··åã¬ãŠã¹ã¢ãã« (Gaussian Mixture Model, GMM) ãå
šè¬çã«åèã«ãªããŸãã
10.4.2 é ã倿°ãšäºåŸç¢ºç
æ··åæ£èŠååžã¢ãã«ã§ã¯ãããŒã¿ããKçµã®æ··åæ¯ãå¹³åãã¯ãã«ãå ±åæ£è¡åãæšå®ããããã«ã1ã€ã®ããŒã¿ãã©ã®ã¯ã©ã¹ã¿ã«å±ããããæšå®ããå¿ èŠããããŸãã1ã€ã®ããŒã¿ãKåã®ã¯ã©ã¹ã¿ã®ã©ãã«å±ãããã衚çŸããK次å 倿°ã䜿ããŸããããã¯One-Hot Encoding衚çŸã§å±ããã¯ã©ã¹ã¿$z_k$ã1ãšãªããŸãã
z = (z_1, \ldots , z_K)^T, \;\;
\sum_{k = 1}^K z_k = 1, \;\;
z = (0, \ldots , 0, 1, 0, \ldots , 0)^T
ãã®å€æ°ã¯ã倿°xãæå±ããé ããã¯ã©ã¹ã¿ãæå®ããŠãããããé ã倿°ãšåŒã°ããŸããããšããšæäŸãããããŒã¿ã«ãªããæ£è§£ã©ãã«ãèªåã§èšå®ãã倿°ãªã®ã§ãé ãããšåŒã¶ã€ã¡ãŒãžã§ããããã
倿°xãšé ã倿°zã®åæååžã¯ãã€ãºã®å®çãå
ããŠä»¥äžã«åè§£ã§ããŸãã
p(x, z) = p(z)p(x \mid z)
ãã®äžã§$p(z_k=1)=\pi_k$ãªã®ã§ã**é ã倿°ã®ååžp(z)**ã¯ä»¥äžã®ããã«è¡šçŸã§ããŸãã
â»ä»¥äžã®åŒã®$z_k$éšåã¯$z_k$ä¹ãšããæå³ã§ã¯ãªãã§ãã
p(z) = \prod_{k=1}^K \pi_k^{z_k}\\
\because \pi_k^{z_k=0}=1
äžèšã¯çè§£ã«æéããããã®ã§ãå°ã詳ãã(æ£ããçè§£ãå°ãèªä¿¡ãªã)ã
| åŒ | 説æ |
|---|---|
| $K=2$ | ã¯ã©ã¹ã¿æ°ã2 |
| $\pi=(\pi_1,\pi_2)=(0.2, 0.8)$ | ã¯ã©ã¹ã¿1ãžã®æå±ç¢ºçã0.2 ã¯ã©ã¹ã¿2ãžã®æå±ç¢ºçã0.8 |
| $z=(1,0)$ | ã¯ã©ã¹ã¿1ã«å±ããŠããå Žåã®é ã倿° æçµçã«äºåŸç¢ºçã§æå°€æšå®ãã |
| $p(z_1=1)=\pi_1^{z_1} \times \pi_2^{z_2}=0.2 \times 1$ | ã¯ã©ã¹ã¿1ã«æå±ãã確çã0.2 |
芳枬ããŒã¿ã®é ã倿°ã«ããæ¡ä»¶ä»ãååžã¯ä»¥äžã®åŒã
p(x | z) = \prod_{k = 1}^K (\mathcal{N}(x | \mu_k, \Sigma_k))^{z_k}\\
\because p(x | z_k = 1) = \mathcal{N}(x | \mu_k, \Sigma_k)
p(x)ã¯åæååžp(x, z)ããã¹ãŠã®zã«ã€ããŠã®ç·åãšãªãã®ã§ä»¥äžã®åŒã
\begin{eqnarray}
p(x)& = & \sum_{k = 1}^K p(z)\\
& = & \sum_{k = 1}^K \prod_{k = 1}^K \pi_k^{z_k} \prod_{k = 1}^K (\mathcal{N}(x | \mu_k, \Sigma_k))^{z_k}\\
& = & \sum_{k = 1}^K \pi_k \mathcal{N}(x|\mu_k , \Sigma_k)\\
\end{eqnarray}
ããã§ãããããæ±ãããã£ãé ã倿°ã®äºåŸç¢ºç$\gamma(z_k)$ãèšç®ã§ããŸãã
\begin{eqnarray}
\gamma(z_k) = p(z_k = 1 | x)& = & \frac{p(z_k = 1) p(x | z_k = 1)}{p(x)}\\
& = & \frac{p(z_k = 1) p(x | z_k = 1)}{\sum_{j = 1}^K p(z_j = 1)p(x | z_j = 1)}\\
& = & \frac{\pi_k \mathcal{N}(x | \mu_k, \Sigma_k)}{\sum_{j = 1}^K \pi_j \mathcal{N}(x | \mu_j, \Sigma_j)}\\
\end{eqnarray}
10.4.3 å®å šããŒã¿ã®å¯Ÿæ°å°€åºŠ
芳枬ããŒã¿ãšé ã倿°ã䜵ããŠå®å šããŒã¿ãšããå®å šããŒã¿ã®å°€åºŠãæå€§ã«ãããã©ã¡ãŒã¿ãæ±ããŸãããã ããæå°€æšå®å€ãçŽæ¥æ±ããããšã¯ã§ããŸããã
10.4.4 Q颿°
å®å šããŒã¿ã®å¯Ÿæ°å°€åºŠã®é ã倿°ã«é¢ããæåŸ å€ããšããšãé ã倿°ã®äºåŸç¢ºçã«ãªããŸããé ã倿°ã®æåŸ å€ãäºåŸç¢ºçã§çœ®ãæãã颿°ãQ颿°ãšåŒã³ãŸãã
10.4.5 EMã¢ã«ãŽãªãºã , 10.4.6 Mã¹ãããã®å°åº, 10.4.7 EMã¢ã«ãŽãªãºã ã®æ§è³ª
ãå
šåããšèšããªãããçç¥ããŸãããªããªããçè§£ã«æéãããããããªåé¢ãä»åŸäœ¿ãã®ãïŒãšæããéšåããããããé£ã°ããŸãã
å¿
èŠãšãªã£ãããã®ãããã®ããŒãžãèŠãŠå匷ããããšæããŸãã
- EMã¢ã«ãŽãªãºã 培åºè§£èª¬
- EMã¢ã«ãŽãªãºã (slideshare)
- æ°åŒã䜿ããã€ã¡ãŒãžã§çè§£ããEMã¢ã«ãŽãªãºã (slideshare)
ãã¯ããã¿ãå匷系èšäºãªã³ã¯
| ç« | é ç® | æé(h) | é£æåºŠ | åŠãã ããš |
|---|---|---|---|---|
| 1 | ã¯ããã« | 8.9 | ç¹åŸŽã®å, ç¹åŸŽç©ºé, 次å ã®åªã | |
| 2 | èå¥èŠåãšåŠç¿æ³ã®æŠèŠ | 12 | ããŒã«ãã¢ãŠãæ³,äº€å·®ç¢ºèªæ³ ,äžã€æãæ³ ,ããŒãã¹ãã©ããæ³ ,ãã€ã¢ã¹ã»åæ£ãã¬ãŒããªã, éåŠç¿ | |
| 3 | ãã€ãºã®èå¥èŠå | 14.8 | ãã€ãºèå¥èŠå, ROCæ²ç· | |
| 4 | 確çã¢ãã«ãšèå¥é¢æ° | 18 |
|
å¹³åãã¯ãã«, å ±åæ£è¡å, æšæºå, ç¡çžé¢å, çœè²å, æ£èŠååž, æå°€æšå® |
| 5 | kæè¿åæ³(kNNæ³) | 8 |
|
æè¿åæ³, ãããã€å¢ç, kNN |
| 6 | ç·åœ¢èå¥é¢æ°(åå) | 30.4 |
|
æ£èŠæ¹çšåŒ |
| 6 | ç·åœ¢èå¥é¢æ°(åŸå) | ååã«æéã¯å«ã |
|
ãã£ãã·ã£ãŒã®ç·åœ¢å€å¥é¢æ°, å€å¥åææ³, ããžã¹ãã£ãã¯ååž° |
| 7 | ããŒã»ãããã³ååŠç¿èŠå | 13.5 |
|
å€å±€ããŒã»ãããã³, 誀差éäŒææ³, ã·ã°ã¢ã€ã颿° |
| 8 | ãµããŒããã¯ãã«ãã·ã³ | 14.7 |
|
ã«ãŒãã«ããªãã¯, Μ-SVM |
| 9 | éšåç©ºéæ³ | 15.4 |
|
äž»æååæ, ç¹ç°å€åè§£, CLAFICæ³, ã«ãŒãã«äž»æååæ, ã«ãŒãã«éšåç©ºéæ³ |
| 10 | ã¯ã©ã¹ã¿ãªã³ã° | 8.4 |
|
è·é¢ã®å ¬ç, ãã³ã³ãã¹ããŒè·é¢, K-平忳, èåæ³, æ··åæ£èŠååžã¢ãã« |
| 11 | è奿©ã®çµã¿åããã«ããæ§èœåŒ·å | 11.6 |
|
ããŒããªãŒã©ã³ãå®ç, æ±ºå®æš, ãã®ã³ã°, ã¢ãããŒã¹ã, ã©ã³ãã ãã©ã¬ã¹ã |