Derniers messages sur Zeste de Savoirhttps://zestedesavoir.com/forums/2015-06-20T13:58:57+02:00Les derniers messages parus sur le forum de Zeste de Savoir.Prévisions étranges avec ARMA, message #622812015-06-20T13:58:57+02:00Vayel/@Vayelhttps://zestedesavoir.com/forums/sujet/3279/previsions-etranges-avec-arma/?page=1#p62281<p>Je prends enfin le temps de détailler.</p>
<hr>
<p>On part de la production quotidienne d'une vache :</p>
<p><figure><img alt="" src="http://zestedesavoir.com/media/galleries/1815/1490f5df-ebd9-41f0-a00d-4f2d79234d99.png.960x960_q85.png"><figcaption>Production quotidienne</figcaption></figure></p>
<p>Mais :</p>
<ul>
<li><a href="http://robjhyndman.com/hyndsight/longseasonality/">R gère mal les longues saisons</a></li>
<li>Il y a beaucoup de bruit</li>
<li>On ne souhaite pas prévoir au jour près</li>
</ul>
<p>On va donc travailler sur des données hebdomadaires :</p>
<p><figure><img alt="" src="http://zestedesavoir.com/media/galleries/1815/2d838d42-25c9-46eb-89d8-d6896c450eb6.png.960x960_q85.png"><figcaption>Production hebdomadaire</figcaption></figure></p>
<p>Puis on utilise R pour calculer notre modèle SARIMA et prévoir :</p>
<p><figure><img alt="" src="http://zestedesavoir.com/media/galleries/1815/b8658bcd-7570-4567-8223-ead863ac230c.png.960x960_q85.png"><figcaption>SARIMA</figcaption></figure></p>
<table class="codehilitetable"><tr><td class="linenos"><div class="linenodiv"><pre> 1
2
3
4
5
6
7
8
9
10
11
12
13</pre></div></td><td class="code"><div class="codehilite"><pre><span class="kn">import</span> <span class="nn">rpy2.robjects</span> <span class="kn">as</span> <span class="nn">robjects</span>
<span class="n">fcast</span> <span class="o">=</span> <span class="n">rpackages</span><span class="o">.</span><span class="n">importr</span><span class="p">(</span><span class="s">'forecast'</span><span class="p">)</span>
<span class="n">ts</span> <span class="o">=</span> <span class="n">robjects</span><span class="o">.</span><span class="n">r</span><span class="p">(</span><span class="s">'ts'</span><span class="p">)</span>
<span class="n">freq</span> <span class="o">=</span> <span class="n">get_season_len</span><span class="p">(</span><span class="n">weekly_data</span><span class="p">)</span>
<span class="n">rdata</span> <span class="o">=</span> <span class="n">robjects</span><span class="o">.</span><span class="n">r</span><span class="p">(</span><span class="s">'as.numeric'</span><span class="p">)(</span><span class="n">weekly_data</span><span class="p">)</span>
<span class="n">rdata</span> <span class="o">=</span> <span class="n">ts</span><span class="p">(</span><span class="n">rdata</span><span class="p">,</span> <span class="n">freq</span><span class="o">=</span><span class="n">freq</span><span class="p">)</span>
<span class="n">fit</span> <span class="o">=</span> <span class="n">fcast</span><span class="o">.</span><span class="n">auto_arima</span><span class="p">(</span><span class="n">rdata</span><span class="p">)</span>
<span class="n">fc</span> <span class="o">=</span> <span class="n">fcast</span><span class="o">.</span><span class="n">forecast</span><span class="p">(</span><span class="n">fit</span><span class="p">,</span> <span class="n">h</span><span class="o">=</span><span class="n">fcast_dur</span><span class="p">)</span>
</pre></div>
</td></tr></table>
<p>C'est mauvais. La saisonnalité semble peu prise en compte. Cela est probablement dû à la période de tarissement, où la production forme un pallier à 0. Je ne l'ai pas fait, mais on pourrait enlever ces palliers et les gérer à part. Là, on utilise la transformée de Fourier, comme indiqué dans le lien précédent :</p>
<p><figure><img alt="" src="http://zestedesavoir.com/media/galleries/1815/2121a72a-0285-4d24-a344-955fe1320e16.png.960x960_q85.png"><figcaption>Fourier + ARIMA</figcaption></figure></p>
<table class="codehilitetable"><tr><td class="linenos"><div class="linenodiv"><pre> 1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19</pre></div></td><td class="code"><div class="codehilite"><pre><span class="k">def</span> <span class="nf">get_best_fit</span><span class="p">(</span><span class="n">dta</span><span class="p">):</span>
<span class="n">best_K</span> <span class="o">=</span> <span class="mi">1</span>
<span class="n">best_fit</span> <span class="o">=</span> <span class="n">fcast</span><span class="o">.</span><span class="n">auto_arima</span><span class="p">(</span><span class="n">dta</span><span class="p">,</span> <span class="n">xreg</span><span class="o">=</span><span class="n">fcast</span><span class="o">.</span><span class="n">fourier</span><span class="p">(</span><span class="n">dta</span><span class="p">,</span> <span class="n">K</span><span class="o">=</span><span class="n">best_K</span><span class="p">),</span> <span class="n">seasonal</span><span class="o">=</span><span class="bp">False</span><span class="p">)</span>
<span class="n">best_aicc</span> <span class="o">=</span> <span class="n">get_aicc</span><span class="p">(</span><span class="n">best_fit</span><span class="p">)</span>
<span class="k">for</span> <span class="n">K</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">2</span><span class="p">,</span> <span class="mi">15</span><span class="p">):</span>
<span class="n">fit</span> <span class="o">=</span> <span class="n">fcast</span><span class="o">.</span><span class="n">auto_arima</span><span class="p">(</span><span class="n">dta</span><span class="p">,</span> <span class="n">xreg</span><span class="o">=</span><span class="n">fcast</span><span class="o">.</span><span class="n">fourier</span><span class="p">(</span><span class="n">dta</span><span class="p">,</span> <span class="n">K</span><span class="o">=</span><span class="n">K</span><span class="p">),</span> <span class="n">seasonal</span><span class="o">=</span><span class="bp">False</span><span class="p">)</span>
<span class="n">aicc</span> <span class="o">=</span> <span class="n">get_aicc</span><span class="p">(</span><span class="n">fit</span><span class="p">)</span>
<span class="k">if</span> <span class="n">aicc</span> <span class="o"><</span> <span class="n">best_aicc</span><span class="p">:</span>
<span class="n">best_fit</span> <span class="o">=</span> <span class="n">fit</span>
<span class="n">best_aicc</span> <span class="o">=</span> <span class="n">aicc</span>
<span class="n">best_K</span> <span class="o">=</span> <span class="n">K</span>
<span class="k">return</span> <span class="n">best_fit</span><span class="p">,</span> <span class="n">best_K</span>
<span class="n">fit</span><span class="p">,</span> <span class="n">K</span> <span class="o">=</span> <span class="n">get_best_fit</span><span class="p">(</span><span class="n">rdata</span><span class="p">)</span>
<span class="n">fc</span> <span class="o">=</span> <span class="n">fcast</span><span class="o">.</span><span class="n">forecast</span><span class="p">(</span><span class="n">fit</span><span class="p">,</span> <span class="n">xreg</span><span class="o">=</span><span class="n">fcast</span><span class="o">.</span><span class="n">fourierf</span><span class="p">(</span><span class="n">rdata</span><span class="p">,</span> <span class="n">K</span><span class="o">=</span><span class="n">K</span><span class="p">,</span> <span class="n">h</span><span class="o">=</span><span class="n">fcast_dur</span><span class="p">))</span>
</pre></div>
</td></tr></table>
<p>C'est bien mieux, même si ce n'est pas l'idéal au niveau de la période de tarissement.</p>Prévisions étranges avec ARMA, message #606252015-06-09T18:00:54+02:00Vayel/@Vayelhttps://zestedesavoir.com/forums/sujet/3279/previsions-etranges-avec-arma/?page=1#p60625<p>As explained <a href="http://robjhyndman.com/hyndsight/longseasonality/">there</a>, the problem with R is due to the size of the seasonal period (about 300 days here).</p>
<p>So I proceeded like <a href="http://robjhyndman.com/hyndsight/forecasting-weekly-data/">that</a>.</p>
<table class="codehilitetable"><tr><td class="linenos"><div class="linenodiv"><pre> 1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61</pre></div></td><td class="code"><div class="codehilite"><pre><span class="kn">import</span> <span class="nn">rpy2.robjects</span> <span class="kn">as</span> <span class="nn">robjects</span>
<span class="kn">import</span> <span class="nn">rpy2.robjects.packages</span> <span class="kn">as</span> <span class="nn">rpackages</span>
<span class="kn">import</span> <span class="nn">matplotlib.pyplot</span> <span class="kn">as</span> <span class="nn">plt</span>
<span class="kn">import</span> <span class="nn">numpy</span> <span class="kn">as</span> <span class="nn">np</span>
<span class="n">fcast</span> <span class="o">=</span> <span class="n">rpackages</span><span class="o">.</span><span class="n">importr</span><span class="p">(</span><span class="s">'forecast'</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">get_aicc</span><span class="p">(</span><span class="n">fit</span><span class="p">):</span>
<span class="k">for</span> <span class="n">item</span> <span class="ow">in</span> <span class="n">fit</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
<span class="k">if</span> <span class="s">'aicc'</span> <span class="ow">in</span> <span class="n">item</span><span class="p">:</span>
<span class="k">return</span> <span class="n">item</span><span class="p">[</span><span class="o">-</span><span class="mi">1</span><span class="p">][</span><span class="mi">0</span><span class="p">]</span>
<span class="k">return</span> <span class="bp">None</span>
<span class="k">def</span> <span class="nf">get_best_fit</span><span class="p">(</span><span class="n">dta</span><span class="p">):</span>
<span class="n">best_K</span> <span class="o">=</span> <span class="mi">1</span>
<span class="n">best_fit</span> <span class="o">=</span> <span class="n">fcast</span><span class="o">.</span><span class="n">auto_arima</span><span class="p">(</span><span class="n">dta</span><span class="p">,</span> <span class="n">xreg</span><span class="o">=</span><span class="n">fcast</span><span class="o">.</span><span class="n">fourier</span><span class="p">(</span><span class="n">dta</span><span class="p">,</span> <span class="n">K</span><span class="o">=</span><span class="n">best_K</span><span class="p">),</span> <span class="n">seasonal</span><span class="o">=</span><span class="bp">False</span><span class="p">)</span>
<span class="n">best_aicc</span> <span class="o">=</span> <span class="n">get_aicc</span><span class="p">(</span><span class="n">best_fit</span><span class="p">)</span>
<span class="k">for</span> <span class="n">K</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">2</span><span class="p">,</span> <span class="mi">15</span><span class="p">):</span>
<span class="n">fit</span> <span class="o">=</span> <span class="n">fcast</span><span class="o">.</span><span class="n">auto_arima</span><span class="p">(</span><span class="n">dta</span><span class="p">,</span> <span class="n">xreg</span><span class="o">=</span><span class="n">fcast</span><span class="o">.</span><span class="n">fourier</span><span class="p">(</span><span class="n">dta</span><span class="p">,</span> <span class="n">K</span><span class="o">=</span><span class="n">K</span><span class="p">),</span> <span class="n">seasonal</span><span class="o">=</span><span class="bp">False</span><span class="p">)</span>
<span class="n">aicc</span> <span class="o">=</span> <span class="n">get_aicc</span><span class="p">(</span><span class="n">fit</span><span class="p">)</span>
<span class="k">if</span> <span class="n">aicc</span> <span class="o"><</span> <span class="n">best_aicc</span><span class="p">:</span>
<span class="n">best_fit</span> <span class="o">=</span> <span class="n">fit</span>
<span class="n">best_aicc</span> <span class="o">=</span> <span class="n">aicc</span>
<span class="n">best_K</span> <span class="o">=</span> <span class="n">K</span>
<span class="k">return</span> <span class="n">best_fit</span><span class="p">,</span> <span class="n">best_K</span>
<span class="k">def</span> <span class="nf">get_fcast_vals</span><span class="p">(</span><span class="n">fc</span><span class="p">):</span>
<span class="sd">"""Only because I don't know how to get </span>
<span class="sd"> them from the R list vector directly.</span>
<span class="sd"> """</span>
<span class="n">fc</span> <span class="o">=</span> <span class="nb">str</span><span class="p">(</span><span class="n">fc</span><span class="p">)</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s">'</span><span class="se">\n</span><span class="s">'</span><span class="p">)</span>
<span class="n">fc</span> <span class="o">=</span> <span class="n">fc</span><span class="p">[</span><span class="mi">1</span><span class="p">:</span><span class="o">-</span><span class="mi">1</span><span class="p">]</span> <span class="c"># Remove labels and empty string</span>
<span class="k">return</span> <span class="p">[</span><span class="nb">float</span><span class="p">(</span><span class="n">line</span><span class="o">.</span><span class="n">split</span><span class="p">()[</span><span class="mi">1</span><span class="p">])</span> <span class="k">for</span> <span class="n">line</span> <span class="ow">in</span> <span class="n">fc</span><span class="p">]</span>
<span class="k">def</span> <span class="nf">main</span><span class="p">():</span>
<span class="n">period</span> <span class="o">=</span> <span class="mi">7</span>
<span class="n">n</span><span class="p">,</span> <span class="n">data</span> <span class="o">=</span> <span class="n">tools</span><span class="o">.</span><span class="n">change_period</span><span class="p">(</span><span class="n">data</span><span class="p">,</span> <span class="n">period</span><span class="p">)</span>
<span class="n">freq</span> <span class="o">=</span> <span class="n">lact_dur</span><span class="o">/</span><span class="n">period</span>
<span class="n">h</span> <span class="o">=</span> <span class="mi">100</span> <span class="c"># Number of forecasts</span>
<span class="n">rdata</span> <span class="o">=</span> <span class="n">data</span><span class="o">.</span><span class="n">tolist</span><span class="p">()</span>
<span class="n">rdata</span> <span class="o">=</span> <span class="n">robjects</span><span class="o">.</span><span class="n">r</span><span class="p">(</span><span class="s">'as.numeric'</span><span class="p">)(</span><span class="n">rdata</span><span class="p">)</span>
<span class="n">rdata</span> <span class="o">=</span> <span class="n">ts</span><span class="p">(</span><span class="n">rdata</span><span class="p">,</span> <span class="n">freq</span><span class="o">=</span><span class="n">freq</span><span class="p">)</span>
<span class="n">fit</span><span class="p">,</span> <span class="n">K</span> <span class="o">=</span> <span class="n">get_best_fit</span><span class="p">(</span><span class="n">rdata</span><span class="p">)</span>
<span class="n">fc</span> <span class="o">=</span> <span class="n">fcast</span><span class="o">.</span><span class="n">forecast</span><span class="p">(</span><span class="n">fit</span><span class="p">,</span> <span class="n">xreg</span><span class="o">=</span><span class="n">fcast</span><span class="o">.</span><span class="n">fourierf</span><span class="p">(</span><span class="n">rdata</span><span class="p">,</span> <span class="n">K</span><span class="o">=</span><span class="n">K</span><span class="p">,</span> <span class="n">h</span><span class="o">=</span><span class="n">h</span><span class="p">))</span>
<span class="n">fc</span> <span class="o">=</span> <span class="n">get_fcast_vals</span><span class="p">(</span><span class="n">fc</span><span class="p">)</span>
<span class="c"># Plot</span>
<span class="n">plt</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="nb">range</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">data</span><span class="p">)),</span> <span class="n">data</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s">'Weekly production'</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="nb">range</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">data</span><span class="p">),</span> <span class="nb">len</span><span class="p">(</span><span class="n">data</span><span class="p">)</span> <span class="o">+</span> <span class="n">h</span><span class="p">),</span> <span class="n">fc</span><span class="p">,</span> <span class="s">'r'</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s">'Forecast'</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">show</span><span class="p">()</span>
</pre></div>
</td></tr></table>
<p><figure><img alt="" src="http://zestedesavoir.com/media/galleries/1815/40c7e510-0625-457e-ad80-460debe3a1d0.png.960x960_q85.png"><figcaption> </figcaption></figure></p>
<p><figure><img alt="" src="http://zestedesavoir.com/media/galleries/1815/2cd14011-7784-4c24-8f50-af3f8c45cb99.png.960x960_q85.png"><figcaption> </figcaption></figure></p>
<p><figure><img alt="" src="http://zestedesavoir.com/media/galleries/1815/87313dfb-f3ca-43d7-8fd5-35770036d116.png.960x960_q85.png"><figcaption> </figcaption></figure></p>
<p><figure><img alt="" src="http://zestedesavoir.com/media/galleries/1815/0353d54f-bdd1-4873-bcaf-ef3d118c9cd5.png.960x960_q85.png"><figcaption> </figcaption></figure></p>
<p><figure><img alt="" src="http://zestedesavoir.com/media/galleries/1815/359446da-f372-4613-9ce5-9c9816cf0ce4.png.960x960_q85.png"><figcaption> </figcaption></figure></p>
<p><figure><img alt="" src="http://zestedesavoir.com/media/galleries/1815/99081313-537b-4678-82d5-334ab7b4379a.png.960x960_q85.png"><figcaption> </figcaption></figure></p>
<p><figure><img alt="" src="http://zestedesavoir.com/media/galleries/1815/0b69b670-1f1b-452b-aacd-c33211132012.png.960x960_q85.png"><figcaption> </figcaption></figure></p>
<p>Not perfect but far better than before.</p>
<hr>
<p>J'expliquerai tout ça en français ultérieurement. N'hésitez pas à me rappeler à l'ordre si nécessaire.</p>Prévisions étranges avec ARMA, message #600972015-06-06T11:28:28+02:00Vayel/@Vayelhttps://zestedesavoir.com/forums/sujet/3279/previsions-etranges-avec-arma/?page=1#p60097<p>The seasonality is not taken into account but I managed to improve the model.</p>
<ul>
<li>Dwindle the number of points and shift the data to apply log</li>
</ul>
<p><figure><img alt="" src="http://zestedesavoir.com/media/galleries/1815/2d838d42-25c9-46eb-89d8-d6896c450eb6.png.960x960_q85.png"><figcaption>Weekly data</figcaption></figure></p>
<table class="codehilitetable"><tr><td class="linenos"><div class="linenodiv"><pre> 1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20</pre></div></td><td class="code"><div class="codehilite"><pre><span class="k">def</span> <span class="nf">change_period</span><span class="p">(</span><span class="n">data</span><span class="p">,</span> <span class="n">period</span><span class="p">):</span>
<span class="sd">"""Change the period of the series. The series is not altered.</span>
<span class="sd"> :param data: The values of the series.</span>
<span class="sd"> :param period: The new period.</span>
<span class="sd"> :type data: list</span>
<span class="sd"> :type period: int</span>
<span class="sd"> :return: The number of periods and the new series.</span>
<span class="sd"> :rtype: tuple</span>
<span class="sd"> """</span>
<span class="n">n</span> <span class="o">=</span> <span class="nb">int</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">data</span><span class="p">)</span><span class="o">/</span><span class="n">period</span><span class="p">)</span>
<span class="n">series</span> <span class="o">=</span> <span class="p">[</span><span class="nb">sum</span><span class="p">(</span><span class="n">data</span><span class="p">[</span><span class="n">i</span><span class="o">*</span><span class="n">period</span><span class="p">:(</span><span class="n">i</span><span class="o">+</span><span class="mi">1</span><span class="p">)</span><span class="o">*</span><span class="n">period</span><span class="p">])</span><span class="o">/</span><span class="n">period</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">n</span><span class="p">)]</span>
<span class="k">return</span> <span class="n">n</span><span class="p">,</span> <span class="n">series</span>
<span class="n">n</span><span class="p">,</span> <span class="n">data</span> <span class="o">=</span> <span class="n">change_period</span><span class="p">(</span><span class="n">data</span><span class="p">,</span> <span class="mi">7</span><span class="p">)</span>
</pre></div>
</td></tr></table>
<ul>
<li>Take the log to stabilize the variance</li>
</ul>
<p><figure><img alt="" src="http://zestedesavoir.com/media/galleries/1815/247c0cf5-a5b5-4fff-8769-8eb339df3d1e.png.960x960_q85.png"><figcaption>Log</figcaption></figure></p>
<ul>
<li>Look for a model <code>forecast.auto.arima</code> (R function)</li>
</ul>
<table class="codehilitetable"><tr><td class="linenos"><div class="linenodiv"><pre> 1
2
3
4
5
6
7
8
9
10</pre></div></td><td class="code"><div class="codehilite"><pre><span class="kn">import</span> <span class="nn">rpy2.robjects</span> <span class="kn">as</span> <span class="nn">robjects</span>
<span class="kn">import</span> <span class="nn">rpy2.robjects.packages</span> <span class="kn">as</span> <span class="nn">rpackages</span>
<span class="n">fcast</span> <span class="o">=</span> <span class="n">rpackages</span><span class="o">.</span><span class="n">importr</span><span class="p">(</span><span class="s">'forecast'</span><span class="p">)</span>
<span class="n">ts</span> <span class="o">=</span> <span class="n">robjects</span><span class="o">.</span><span class="n">r</span><span class="p">(</span><span class="s">'ts'</span><span class="p">)</span>
<span class="n">rdata</span> <span class="o">=</span> <span class="n">robjects</span><span class="o">.</span><span class="n">r</span><span class="p">(</span><span class="s">'as.numeric'</span><span class="p">)(</span><span class="n">log_data</span><span class="p">)</span>
<span class="n">rdata</span> <span class="o">=</span> <span class="n">ts</span><span class="p">(</span><span class="n">rdata</span><span class="p">)</span>
<span class="n">model</span> <span class="o">=</span> <span class="n">fcast</span><span class="o">.</span><span class="n">auto_arima</span><span class="p">(</span><span class="n">rdata</span><span class="p">,</span> <span class="n">stepwise</span><span class="o">=</span><span class="bp">False</span><span class="p">,</span> <span class="n">approximation</span><span class="o">=</span><span class="bp">False</span><span class="p">,</span> <span class="n">seasonal</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span>
<span class="c"># Returns: ARIMA(2, 0, 2)</span>
</pre></div>
</td></tr></table>
<ul>
<li>Fit the model to the log data</li>
</ul>
<p><figure><img alt="" src="http://zestedesavoir.com/media/galleries/1815/1ef2c3d7-97b4-4d99-94b0-63bbe1d0c90a.png.960x960_q85.png"><figcaption>Forecasts - ARIMA(2, 0, 2)</figcaption></figure></p>
<table class="codehilitetable"><tr><td class="linenos"><div class="linenodiv"><pre> 1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16</pre></div></td><td class="code"><div class="codehilite"><pre><span class="k">def</span> <span class="nf">arma</span><span class="p">(</span><span class="n">data</span><span class="p">,</span> <span class="n">p</span><span class="p">,</span> <span class="n">d</span><span class="p">,</span> <span class="n">q</span><span class="p">,</span> <span class="n">start</span><span class="p">,</span> <span class="n">end</span><span class="p">):</span>
<span class="n">model</span> <span class="o">=</span> <span class="n">sm</span><span class="o">.</span><span class="n">tsa</span><span class="o">.</span><span class="n">ARIMA</span><span class="p">(</span><span class="n">data</span><span class="p">,</span> <span class="p">(</span><span class="n">p</span><span class="p">,</span> <span class="n">d</span><span class="p">,</span> <span class="n">q</span><span class="p">))</span><span class="o">.</span><span class="n">fit</span><span class="p">()</span>
<span class="k">if</span> <span class="n">d</span><span class="p">:</span> <span class="c"># ARIMA model</span>
<span class="n">predict</span> <span class="o">=</span> <span class="n">model</span><span class="o">.</span><span class="n">predict</span><span class="p">(</span><span class="n">start</span><span class="p">,</span> <span class="n">end</span><span class="p">,</span> <span class="n">typ</span><span class="o">=</span><span class="s">'levels'</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span> <span class="c"># ARMA model</span>
<span class="n">predict</span> <span class="o">=</span> <span class="n">model</span><span class="o">.</span><span class="n">predict</span><span class="p">(</span><span class="n">start</span><span class="p">,</span> <span class="n">end</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="nb">range</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">data</span><span class="p">)),</span> <span class="n">data</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="nb">range</span><span class="p">(</span><span class="n">start</span><span class="p">,</span> <span class="n">end</span><span class="o">+</span><span class="mi">1</span><span class="p">),</span> <span class="n">predict</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">savefig</span><span class="p">(</span><span class="s">'experiments/arima({}, {}, {}).png'</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">p</span><span class="p">,</span> <span class="n">d</span><span class="p">,</span> <span class="n">q</span><span class="p">))</span>
<span class="n">plt</span><span class="o">.</span><span class="n">clf</span><span class="p">()</span>
<span class="n">fcast_dur</span> <span class="o">=</span> <span class="mi">50</span>
<span class="n">arma</span><span class="p">(</span><span class="n">log_data</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">log_data</span><span class="p">)</span> <span class="o">-</span> <span class="n">fcast_dur</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">log_data</span><span class="p">)</span> <span class="o">+</span> <span class="n">fcast_dur</span><span class="p">)</span>
</pre></div>
</td></tr></table>
<hr>
<p>About the seasonality, I found <a href="http://stats.stackexchange.com/questions/65585/auto-arima-does-not-recognize-seasonal-pattern">that</a>.</p>Prévisions étranges avec ARMA, message #600262015-06-05T18:55:15+02:00Vayel/@Vayelhttps://zestedesavoir.com/forums/sujet/3279/previsions-etranges-avec-arma/?page=1#p60026<p>statsmodels gives the same forecasts as R with an ARIMA(3, 1, 2) model.</p>
<p><figure><img alt="" src="http://zestedesavoir.com/media/galleries/1815/3f35d014-3f49-4cf8-aa5e-dec594575115.png.960x960_q85.png"><figcaption>ARIMA(3, 1, 2)</figcaption></figure></p>
<p>But I cannot figure out why the seasonality does not appear in the automatic model. I even tried an <a href="https://www.otexts.org/fpp/6/5">STL decomposition</a> with R but it gave me an error: <code>la série n'est pas périodique ou elle a moins de deux périodes</code>. In English: <code>the series is not seasonal or has fewer than two seasons</code>. Yet, it appears to have four.</p>
<p>Thanks.</p>Prévisions étranges avec ARMA, message #599492015-06-05T12:30:54+02:00Vayel/@Vayelhttps://zestedesavoir.com/forums/sujet/3279/previsions-etranges-avec-arma/?page=1#p59949<p>It appears to work with R but statsmodels still returns strange forecasts.</p>
<table class="codehilitetable"><tr><td class="linenos"><div class="linenodiv"><pre> 1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21</pre></div></td><td class="code"><div class="codehilite"><pre><span class="kn">import</span> <span class="nn">rpy2.robjects</span> <span class="kn">as</span> <span class="nn">robjects</span>
<span class="kn">import</span> <span class="nn">rpy2.robjects.packages</span> <span class="kn">as</span> <span class="nn">rpackages</span>
<span class="n">fcast</span> <span class="o">=</span> <span class="n">rpackages</span><span class="o">.</span><span class="n">importr</span><span class="p">(</span><span class="s">'forecast'</span><span class="p">)</span>
<span class="n">tseries</span> <span class="o">=</span> <span class="n">rpackages</span><span class="o">.</span><span class="n">importr</span><span class="p">(</span><span class="s">'tseries'</span><span class="p">)</span>
<span class="n">plot</span> <span class="o">=</span> <span class="n">robjects</span><span class="o">.</span><span class="n">r</span><span class="p">(</span><span class="s">'plot'</span><span class="p">)</span>
<span class="n">_list</span> <span class="o">=</span> <span class="n">robjects</span><span class="o">.</span><span class="n">r</span><span class="p">(</span><span class="s">'list'</span><span class="p">)</span>
<span class="n">c</span> <span class="o">=</span> <span class="n">robjects</span><span class="o">.</span><span class="n">r</span><span class="p">(</span><span class="s">'c'</span><span class="p">)</span>
<span class="n">diff</span> <span class="o">=</span> <span class="n">robjects</span><span class="o">.</span><span class="n">r</span><span class="p">(</span><span class="s">'diff'</span><span class="p">)</span>
<span class="n">ts</span> <span class="o">=</span> <span class="n">robjects</span><span class="o">.</span><span class="n">r</span><span class="p">(</span><span class="s">'ts'</span><span class="p">)</span>
<span class="c"># dta is a list containing the productions of one cow</span>
<span class="n">dta</span> <span class="o">=</span> <span class="n">robjects</span><span class="o">.</span><span class="n">r</span><span class="p">(</span><span class="s">'as.numeric'</span><span class="p">)(</span><span class="n">dta</span><span class="p">)</span>
<span class="n">model</span> <span class="o">=</span> <span class="n">fcast</span><span class="o">.</span><span class="n">auto_arima</span><span class="p">(</span><span class="n">ts</span><span class="p">(</span><span class="n">dta</span><span class="p">),</span> <span class="n">stepwise</span><span class="o">=</span><span class="bp">False</span><span class="p">,</span>
<span class="n">approximation</span><span class="o">=</span><span class="bp">False</span><span class="p">)</span>
<span class="c"># The previous line gives an ARIMA(3, 1, 2)</span>
<span class="c"># The seasonnality seems not to be taken into account</span>
<span class="n">fit</span> <span class="o">=</span> <span class="n">fcast</span><span class="o">.</span><span class="n">Arima</span><span class="p">(</span><span class="n">dta</span><span class="p">,</span> <span class="n">order</span><span class="o">=</span><span class="n">c</span><span class="p">(</span><span class="mi">3</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">))</span>
<span class="n">forecast</span> <span class="o">=</span> <span class="n">fcast</span><span class="o">.</span><span class="n">forecast_Arima</span><span class="p">(</span><span class="n">fit</span><span class="p">,</span> <span class="mi">30</span><span class="p">)</span>
<span class="n">plot</span><span class="p">(</span><span class="n">forecast</span><span class="p">)</span>
</pre></div>
</td></tr></table>
<p><figure><img alt="" src="http://zestedesavoir.com/media/galleries/1815/0e8de97a-7709-4577-91c8-89d32203a424.png.960x960_q85.png"><figcaption>ARIMA(3, 1, 2) with R</figcaption></figure></p>
<p>Will work on a seasonal model these days.</p>Processus indiqué comme stationnaire alors que non, message #590882015-05-30T19:08:06+02:00Vayel/@Vayelhttps://zestedesavoir.com/forums/sujet/3275/processus-indique-comme-stationnaire-alors-que-non/?page=1#p59088<blockquote>
<p>La trajectoire n'a pas de tendance. </p>
</blockquote>
<p>Oui.</p>
<blockquote>
<p>La composante saisonnière est régulière en amplitude et en période.</p>
</blockquote>
<p>Oui.</p>
<blockquote>
<p>Tu ne peux pas juger de la stationnarité en une seule réalisation, mais cette réalisation ne discrédite pas l'hypothèse, ce qui est déjà pas mal.</p>
</blockquote>
<p>Pourquoi ? Si <span>$(X_{t})$</span> est mon processus, il semblerait que <span>$X_{t}$</span> et <span>$X_{t+320/2}$</span> n'aient pas la même espérance (la première est dans une zone de forte production, la seconde dans une zone de faible production. Ou inversement.).</p>Processus indiqué comme stationnaire alors que non, message #590842015-05-30T18:58:07+02:00KFC/@KFChttps://zestedesavoir.com/forums/sujet/3275/processus-indique-comme-stationnaire-alors-que-non/?page=1#p59084<p>La trajectoire n'a pas de tendance. La composante saisonnière est régulière en amplitude et en période.
Tu ne peux pas juger de la stationnarité en une seule réalisation, mais cette réalisation ne discrédite pas l'hypothèse, ce qui est déjà pas mal. Depuis, il n'y a pas de raison physique pour que les trajectoires s'écartent trop de celle-ci et donc on peut vraisemblablement affirmer que le processus sous-jacent est stationnaire.</p>Processus indiqué comme stationnaire alors que non, message #590722015-05-30T18:25:25+02:00Vayel/@Vayelhttps://zestedesavoir.com/forums/sujet/3275/processus-indique-comme-stationnaire-alors-que-non/?page=1#p59072<figure><blockquote>
<p>@Gabbro, si tu fais reference au modele <a href="http://zestedesavoir.com/forums/sujet/3279/previsions-etranges-avec-arma/">ici</a>, il semble tout a fait stationnaire au contraire.</p>
</blockquote>
<figcaption><p><a href="http://zestedesavoir.com/forums/sujet/3275/processus-indique-comme-stationnaire-alors-que-non/?page=1#p58848">Höd</a></p></figcaption></figure><p>Pourrais-tu préciser comment tu constates ça ? Le test de Dickey-Fuller semble le confirmer mais :</p>
<ul>
<li>L'autocorrélogramme a une tête bizarre pour une série stationnaire ;</li>
<li>En principe, le processus sous-jacent ne l'est pas vu qu'une vache produit clairement moins de lait en fin de lactation qu'au milieu. D'ailleurs, les périodes de production nulle (périodes de tarissement) à intervalles réguliers semblent poser problème à la stationnarité. Ce qui se retrouve sur le graphe des données différenciées.</li>
</ul>
<p><img alt="" src="http://zestedesavoir.com/media/galleries/1815/b89ea0ed-b4f1-40ce-9b90-000a71313023.png.960x960_q85.png"></p>
<p>D'ailleurs, comment peut-on juger de la stationnarité d'un processus à partir d'une seule réalisation ? Si je prends une réalisation d'un bruit blanc (processus stationnaire), il n'est pas impossible que j'obtienne une droite de pente non nulle. Le cas échéant, j'aurais tendance à dire que le processus n'est pas stationnaire.</p>
<p>Merci. <img alt="^^" src="/static/smileys/hihi.png"></p>Prévisions étranges avec ARMA, message #590132015-05-30T12:18:31+02:00Vayel/@Vayelhttps://zestedesavoir.com/forums/sujet/3279/previsions-etranges-avec-arma/?page=1#p59013<p>That's strange: passing the <code>exog</code> parameter does not change anything.</p>
<p><img alt="" src="http://zestedesavoir.com/media/galleries/1815/c10c37ef-9244-4390-af56-d2cb23d047e3.png.960x960_q85.png"></p>
<table class="codehilitetable"><tr><td class="linenos"><div class="linenodiv"><pre> 1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24</pre></div></td><td class="code"><div class="codehilite"><pre><span class="kn">import</span> <span class="nn">statsmodels.api</span> <span class="kn">as</span> <span class="nn">sm</span>
<span class="k">def</span> <span class="nf">arma</span><span class="p">(</span><span class="n">data</span><span class="p">,</span> <span class="n">p</span><span class="p">,</span> <span class="n">d</span><span class="p">,</span> <span class="n">q</span><span class="p">,</span> <span class="n">start</span><span class="p">,</span> <span class="n">end</span><span class="p">,</span> <span class="n">exog</span><span class="p">):</span>
<span class="n">model</span> <span class="o">=</span> <span class="n">sm</span><span class="o">.</span><span class="n">tsa</span><span class="o">.</span><span class="n">ARIMA</span><span class="p">(</span><span class="n">data</span><span class="p">,</span> <span class="p">(</span><span class="n">p</span><span class="p">,</span> <span class="n">d</span><span class="p">,</span> <span class="n">q</span><span class="p">))</span><span class="o">.</span><span class="n">fit</span><span class="p">()</span>
<span class="n">predict</span> <span class="o">=</span> <span class="n">model</span><span class="o">.</span><span class="n">predict</span><span class="p">(</span><span class="n">start</span><span class="o">=</span><span class="n">start</span><span class="p">,</span> <span class="n">end</span><span class="o">=</span><span class="n">end</span><span class="p">,</span> <span class="n">typ</span><span class="o">=</span><span class="s">'levels'</span><span class="p">,</span> <span class="n">exog</span><span class="o">=</span><span class="n">exog</span><span class="p">,</span>
<span class="n">dynamic</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="nb">range</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">data</span><span class="p">)),</span> <span class="n">data</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="nb">range</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">predict</span><span class="p">)),</span> <span class="n">predict</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">savefig</span><span class="p">(</span><span class="s">'./arma_({}, {}, {}).png'</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">p</span><span class="p">,</span> <span class="n">d</span><span class="p">,</span> <span class="n">q</span><span class="p">))</span>
<span class="n">dta</span> <span class="o">=</span> <span class="n">mongo</span><span class="o">.</span><span class="n">data</span><span class="p">(</span><span class="n">_id</span><span class="p">)</span>
<span class="n">diff_id</span> <span class="o">=</span> <span class="n">diff_ids</span><span class="p">[</span><span class="n">LABELS</span><span class="p">[</span><span class="s">'values'</span><span class="p">]]</span>
<span class="n">diff_dta</span> <span class="o">=</span> <span class="n">mongo</span><span class="o">.</span><span class="n">data</span><span class="p">(</span><span class="n">diff_id</span><span class="p">)</span>
<span class="n">durations</span> <span class="o">=</span> <span class="p">[</span><span class="nb">len</span><span class="p">(</span><span class="n">mongo</span><span class="o">.</span><span class="n">prods</span><span class="p">(</span><span class="n">cow</span><span class="p">,</span> <span class="n">lact</span><span class="p">))</span> <span class="k">for</span> <span class="n">lact</span> <span class="ow">in</span> <span class="n">mongo</span><span class="o">.</span><span class="n">lacts</span><span class="p">(</span><span class="n">cow</span><span class="p">)]</span>
<span class="n">lact_dur</span> <span class="o">=</span> <span class="nb">int</span><span class="p">(</span><span class="nb">sum</span><span class="p">(</span><span class="n">durations</span><span class="p">)</span><span class="o">/</span><span class="nb">len</span><span class="p">(</span><span class="n">durations</span><span class="p">))</span>
<span class="n">exog</span> <span class="o">=</span> <span class="nb">range</span><span class="p">(</span><span class="n">lact_dur</span><span class="p">)</span>
<span class="n">start</span> <span class="o">=</span> <span class="mi">50</span>
<span class="n">end</span> <span class="o">=</span> <span class="mi">1800</span>
<span class="n">arma</span><span class="p">(</span><span class="n">dta</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="n">start</span><span class="p">,</span> <span class="n">end</span><span class="p">,</span> <span class="n">exog</span><span class="p">)</span>
</pre></div>
</td></tr></table>
<p>The series appears to be stationary:</p>
<table class="codehilitetable"><tr><td class="linenos"><div class="linenodiv"><pre>1
2
3
4
5</pre></div></td><td class="code"><div class="codehilite"><pre><span class="o">>>></span> <span class="n">sm</span><span class="o">.</span><span class="n">tsa</span><span class="o">.</span><span class="n">stattools</span><span class="o">.</span><span class="n">adfuller</span><span class="p">(</span><span class="n">dta</span><span class="p">)</span>
<span class="p">(</span><span class="o">-</span><span class="mf">3.8535137605866305</span><span class="p">,</span> <span class="mf">0.0024048823206302677</span><span class="p">,</span> <span class="mi">23</span><span class="p">,</span> <span class="mi">1483</span><span class="p">,</span> <span class="p">{</span><span class="s">'5%'</span><span class="p">:</span> <span class="o">-</span><span class="mf">2.8634908922653302</span><span class="p">,</span> <span class="s">'1%'</span><span class="p">:</span> <span class="o">-</span><span class="mf">3.4347671645756304</span><span class="p">,</span> <span class="s">'10%'</span><span class="p">:</span> <span class="o">-</span><span class="mf">2.5678086339403325</span><span class="p">},</span> <span class="mf">7982.1399270341799</span><span class="p">)</span>
<span class="o">>>></span> <span class="n">sm</span><span class="o">.</span><span class="n">tsa</span><span class="o">.</span><span class="n">stattools</span><span class="o">.</span><span class="n">adfuller</span><span class="p">(</span><span class="n">diff_dta</span><span class="p">)</span>
<span class="p">(</span><span class="o">-</span><span class="mf">7.8470556885881599</span><span class="p">,</span> <span class="mf">5.7318468331677856e-12</span><span class="p">,</span> <span class="mi">20</span><span class="p">,</span> <span class="mi">1485</span><span class="p">,</span> <span class="p">{</span><span class="s">'5%'</span><span class="p">:</span> <span class="o">-</span><span class="mf">2.8634882621786728</span><span class="p">,</span> <span class="s">'1%'</span><span class="p">:</span> <span class="o">-</span><span class="mf">3.4347612052013901</span><span class="p">,</span> <span class="s">'10%'</span><span class="p">:</span> <span class="o">-</span><span class="mf">2.5678072333888831</span><span class="p">},</span> <span class="mf">7985.3887776714364</span><span class="p">)</span>
</pre></div>
</td></tr></table>Processus indiqué comme stationnaire alors que non, message #590072015-05-30T11:26:41+02:00adri1/@adri1https://zestedesavoir.com/forums/sujet/3275/processus-indique-comme-stationnaire-alors-que-non/?page=1#p59007<blockquote>
<p>@Höd : pour moi, stationnaire signifie qu'aucune variable associé au système n'a de dépendance au temps.</p>
</blockquote>
<p>Ça, c'est la définition physique la plus basique de la stationnarité (dérivées temporelles nulle). Mais là, il s'agit d'un processus qu'on qualifie en général en physique de <em>statistiquement</em> stationnaire (tu as sûrement déjà croisé le terme).</p>Processus indiqué comme stationnaire alors que non, message #589372015-05-29T19:49:25+02:00KFC/@KFChttps://zestedesavoir.com/forums/sujet/3275/processus-indique-comme-stationnaire-alors-que-non/?page=1#p58937<p>Ce n'est pas la définition de stationnaire.
La stationnarité est une propriété du processus qui génère les trajectoires.</p>
<p>Un processus est stationnaire si son espérance est indépendante du temps, sa variance est finie et indépendante du temps et l'auto-corrélation entre deux VA ne dépend que de la distance temporelle entre les deux (et donc l'auto-corrélation dépend du temps).</p>
<p><a href="http://images.slideplayer.fr/5/1619648/slides/slide_7.jpg">Un exemple</a></p>
<p>Cela n'inderdit pas des périodicités. Le processus <span>$X(t) = a\text{sin}(\omega x + \Phi)$</span> est stationnaire pourtant tu conviendras qu'il s'autorise une périodicité temporelle.</p>Processus indiqué comme stationnaire alors que non, message #589302015-05-29T19:21:39+02:00Gabbro/@Gabbrohttps://zestedesavoir.com/forums/sujet/3275/processus-indique-comme-stationnaire-alors-que-non/?page=1#p58930<p>@Höd : pour moi, stationnaire signifie qu'aucune variable associé au système n'a de dépendance au temps. Là, il y a des creux, des bosses, des plats… Autour de 1400, ça vaut 0, autour de 800, ça vaut pas du tout 0.</p>
<p>Après, selon les domaines, on met parfois des sens différents (mais proche) sur les même mots. Si tu veux dire qu'il est équivalent de prendre une année ou une autre (0 -> 365 ou 1095 -> 1460) (aux variations intrinsèques près), là, on est d'accord.</p>Prévisions étranges avec ARMA, message #589202015-05-29T18:31:54+02:00josef.pktd/@josef.pktdhttps://zestedesavoir.com/forums/sujet/3279/previsions-etranges-avec-arma/?page=1#p58920<p>The seasonal trend would be stacking together the numbers from one to about 320, once for each season, exactly the way the x-axis is labeled in the last plot.</p>
<p>The way I thought about this is that <code>day</code> is your concatenated index representing days on record for the cow, say 800 in the previous plot, start of corresponding season is at 700, so the trend period for day 800 would be 100, and would be the same as in your last plot (days since beginning of season).</p>
<p>Adding trend like this would largely correspond to fitting a regression line or curve to your last plot, production as function of day of season pooled over 5 years).</p>Processus indiqué comme stationnaire alors que non, message #589072015-05-29T16:59:30+02:00KFC/@KFChttps://zestedesavoir.com/forums/sujet/3275/processus-indique-comme-stationnaire-alors-que-non/?page=1#p58907<p>Les saisonnalites n'ont rien a voir avec la stationarite lorsqu'elle sont regulieres comme c'est le cas ici.</p>Prévisions étranges avec ARMA, message #589022015-05-29T16:51:09+02:00Vayel/@Vayelhttps://zestedesavoir.com/forums/sujet/3279/previsions-etranges-avec-arma/?page=1#p58902<p>I am sorry, I did not understand what <code>day</code> is. Does the arange represent the days for the season to predict? Else, whose season are they?</p>
<p>The seasons appear to last the same time:</p>
<p><figure><img alt="" src="http://zestedesavoir.com/media/galleries/1815/d55d7637-1df0-44ea-b18e-fb31b01c63e1.png.960x960_q85.png"><figcaption>Lactations</figcaption></figure></p>Prévisions étranges avec ARMA, message #588982015-05-29T16:40:48+02:00josef.pktd/@josef.pktdhttps://zestedesavoir.com/forums/sujet/3279/previsions-etranges-avec-arma/?page=1#p58898<p><code>exog</code> can be any explanatory variables, the automatically included intercept and trend options are similar to it.
<code>exog</code> doesn't have to be a real explanatory variable like feed, but can also just be a dummy variable or trend variable to adjust the mean of the prediction. </p>
<p>You would need to provide the <code>exog</code> for out of sample prediction.
exog could just be something like <code>np.arange(day - start_of_season, end_of_season - start_of_season)</code> and you give the corresponding slice for the forecasting period to <code>predict</code>.</p>
<p>In general, the model is that the deviation from the deterministic part is an ARIMA process (y - X * beta) is ARIMA. The exog, X, could be empty, that is have zero intercept, just a constant, or a trend or anything else, like a within season trend.</p>Processus indiqué comme stationnaire alors que non, message #588972015-05-29T16:39:43+02:00Vayel/@Vayelhttps://zestedesavoir.com/forums/sujet/3275/processus-indique-comme-stationnaire-alors-que-non/?page=1#p58897<p>Le <a href="http://statsmodels.sourceforge.net/0.5.0/generated/statsmodels.tsa.stattools.adfuller.html">test augmenté de Dickey-Fuller</a> semble le confirmer :</p>
<table class="codehilitetable"><tr><td class="linenos"><div class="linenodiv"><pre>1
2
3</pre></div></td><td class="code"><div class="codehilite"><pre>ADF crude: (-3.8535137605866305, 0.0024048823206302677, 23, 1483, {'5%': -2.8634908922653302, '10%': -2.5678086339403325, '1%': -3.4347671645756304}, 7982.1399270341799)
ADF diff 1: (-7.8470556885881599, 5.7318468331677856e-12, 20, 1485, {'5%': -2.8634882621786728, '10%': -2.5678072333888831, '1%': -3.4347612052013901}, 7985.3887776714364)
</pre></div>
</td></tr></table>
<p>Mais pourtant j'ai un corrélogramme qui semble indiquer que ce n'est pas le cas :</p>
<p><figure><img alt="" src="http://zestedesavoir.com/media/galleries/1815/312e854a-54b3-4c5a-acb6-71e6e95218f7.png.960x960_q85.png"><figcaption>ACF</figcaption></figure></p>
<p>Si c'est effectivement stationnaire, comment je sélectionne mon paramètre <span>$q$</span> pour mon mondèle ARMA ? C'est censé être le dernier pic non nul, mais là…</p>
<p>Enfin, comme j'ai des saisonnalités, ça ne peut pas l'être, si ?</p>Prévisions étranges avec ARMA, message #588812015-05-29T15:51:44+02:00Vayel/@Vayelhttps://zestedesavoir.com/forums/sujet/3279/previsions-etranges-avec-arma/?page=1#p58881<p>Oh! You're right! I missed it! In this <a href="http://www.analyticsvidhya.com/blog/2015/03/framework-application-build-arima-model/">example</a>, the model is made on one season only.</p>
<p>However, I do not really understand what ARMAX is. <code>exog</code> means that I am trying to explain the production with another variable (eg: feed), doesn't it? But I have no other variable.</p>
<p>Moreover, <code>exog</code> is an array in <code>ARMA.predict</code>. Have I to pass the values of the series during one lactation (I primarily thought that I had to pass the duration of a period)?</p>Prévisions étranges avec ARMA, message #588772015-05-29T15:36:36+02:00josef.pktd/@josef.pktdhttps://zestedesavoir.com/forums/sujet/3279/previsions-etranges-avec-arma/?page=1#p58877<p>Ok, it wasn't clear to me how you combine different lactation "seasons"</p>
<p>The problem, as briefly discussed in the other thread, is that if you run ARIMA on the entire sample (4 seasons like in the last plot), the drift or intercept in differences is estimated across seasons not within a season. (it's positive in the last plot, I guess)</p>
<p>So, using the current ARIMAX implementation, you cannot estimate a within season intercept directly.
There is a seasonal SARIMAX model in statsmodels master, but I think it will be easier to model the seasonal trend directly. Try exog = period_in_season to see if it works better and then you could try a spline with a few knots (degrees of freedom)
<a href="http://patsy.readthedocs.org/en/latest/spline-regression.html">http://patsy.readthedocs.org/en/latest/spline-regression.html</a></p>Prévisions étranges avec ARMA, message #588652015-05-29T14:47:26+02:00Vayel/@Vayelhttps://zestedesavoir.com/forums/sujet/3279/previsions-etranges-avec-arma/?page=1#p58865<p>The problem is there when there are no data to build the prediction. Indeed, with <code>dynamic</code> to <code>True</code>, I have:</p>
<p><figure><img alt="" src="http://zestedesavoir.com/media/galleries/1815/c5b37500-5600-473e-9cc4-4b1da3f02dc8.png.960x960_q85.png"><figcaption>Dynamic prediction</figcaption></figure></p>
<p>Linear prediction:</p>
<p><figure><img alt="" src="http://zestedesavoir.com/media/galleries/1815/dc39d6e5-63c3-4ef3-a4c7-560b05c7f624.png.960x960_q85.png"><figcaption>Linear prediction</figcaption></figure></p>
<p>Levels prediction (default in <code>plot_predict</code> : https://github.com/statsmodels/statsmodels/blob/master/statsmodels/tsa/arima_model.py#L1834):</p>
<p><figure><img alt="" src="http://zestedesavoir.com/media/galleries/1815/2cea32ff-0765-4de5-8d5b-e546e1c24f8a.png.960x960_q85.png"><figcaption>Levels prediction</figcaption></figure></p>
<p>What's strange is that ARIMA works well <a href="http://www.analyticsvidhya.com/blog/2015/03/framework-application-build-arima-model/">here</a>. Yet, I followed the same steps.</p>
<p>Concerning the differences, I assume that the seasons depend on the cow. Anyway, I first work on one cow only and lead a univariate analysis (eg: feed influence is not studied). But honestly I do not know how it works. Modelling is one of the targets.</p>Prévisions étranges avec ARMA, message #588572015-05-29T14:21:58+02:00josef.pktd/@josef.pktdhttps://zestedesavoir.com/forums/sujet/3279/previsions-etranges-avec-arma/?page=1#p58857<p>According to the documentation and the code the default for <code>typ</code> is <code>linear</code> not <code>level</code>.</p>
<p>about the trend:</p>
<p>I think either differencing or explicit modeling of it would work.
However, if you difference the series, then there should be a negative intercept, which should result in a downward drift or stochastic trend when you predict in levels.</p>
<p>If you have a relatively regular trend in the seasonal structure (lactation period), then I think modeling it as fixed trend would work better. You could use a low order polynomial or a low order spline (from patsy) to model curvature in the within season trend.</p>
<p>I browsed some of your plots: Are the differences in the seasonal trend between cows inherent to the cows (differences across individuals) or are there some explanatory variables for it.</p>
<p>In general, since you are doing long term forecasting (many periods, days), then it might be useful to think more about trend and deterministic components, than the short term time series behavior with ARIMA with small number of lags.
Although, differencing might work quite well for prediction when there is break in the seasonal pattern for a cow, which I guess is caused by some specific events (based on looking at some of your plots).</p>