> For the complete documentation index, see [llms.txt](https://sliu583.gitbook.io/blog/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://sliu583.gitbook.io/blog/specific-work/wisr-group/cache/index/important-sampling.md).

# Important Sampling

### Monte Carlo 

* Repeated random sampling 
* RL: estimate directly from experiences 
* DP
  * Agent knows the transition probabilities 
* Monte Carlo: Estimate values by averaging over a large number of random samples 

![](/files/-MkB3WlTyIJKs3SuCPDo)

![](/files/-MkB3be5Rj8Bc0TEwhk2)

![](/files/-MkB3dcK0JpeCVRAglzB)

### Epsilon-soft policies 

![](/files/-MkBJ2cjnL6a5QHhvssr)

* Continuously explore 
* Non-zero probability to each action in every state 
* Always stochastic (probability) 

![](/files/-MkBJSQwKT7QS9be7jPe)

![](/files/-MkBJnJWFtitSppA0GXZ)

![](/files/-MkBJvpDLPLx01_uocvr)