智能工具库

How to Build a Markov Chain in Python With NumPy

How to Build a Markov Chain in Python With NumPy
Let's start with a familiar example: sitting down to write an article. You've got an idea, you've got a basic structure, you might even have a deadlin
0来源:Towards Data Science

本文由「How to Build a Markov Chain in Python With NumPy」的公开内容整理,完整内容与后续更新请点击文末链接阅读原文。

Let's start with a familiar example: sitting down to write an article. You've got an idea, you've got a basic structure, you might even have a deadline. You start writing. The next day you might well be writing some more, you might be editing, you might be completely distracted by something else entirely. A bit of research becomes a whole lot of research. Before you know it you've got seventeen browser tabs open, three potential titles, and absolutely nothing at all.Now what if we could actually describe this very erratic process in mathematical terms?

What if we could work out the probability of being productive tomorrow given what we're doing today?

This is where Markov chains become interesting.A Markov chain is a mathematical model for a system that evolves from one state to another over time. Given the current state, the past is irrelevant in terms of predicting the next state. That's the Markov property.In this article, we're going to look at Markov chains using a small fictional writing exercise example. We'll look at transition matrices, work out multi-step probabilities, look at stationary distributions and then go on to actually implement a Markov chain in Python.We'll also talk about something I think deserves a bit more attention: just because we can model something as a Markov chain doesn't mean we necessarily should.1. The Writing Experiment: Three States, Many PossibilitiesLet's imagine a writer working on an article over several days. Each day, we classify their activity into one of three mutually exclusive states:Drafting (D): Actively writing new material.Revising (R): Editing and improving existing material.Distracted (X): Doing practically everything except working on the article.Image created by Author using CanvaTo keep the example manageable, suppose we observe one state per day, representing the writer's primary activity. Now imagine we have estimated the following transition probabilities:Today's stateDrafting tomorrowRevising tomorrowDistracted tomorrowDrafting60%30%10%Revising30%50%20%Distracted20%40%40%Let's interpret the first row of the table. If the writer is drafting today, there's a 60% chance they'll continue drafting tomorrow, a 30% chance they'll move to revising, and a 10% chance they'll become distracted. Interestingly, if they are distracted today, there's still a 60% combined chance of coming back to drafting or revising tomorrow. Procrastination may not be quite as permanent as we sometimes think.The key point is that we've described the writer's behavior in terms of states and the probabilities of transitioning between those states. That's the basis of a Markov chain.2. The Markov Property: Does Yesterday Really Not Matter?

The defining feature of a first-order Markov chain is its memoryless property.Mathematically:The Markov PropertyHere, (X_t) represents the state of the process at time (t), while (i) and (j) represent possible states.Don't worry if the notation looks intimidating. The idea is simpler than it appears.Suppose our writer is drafting today. According to the transition probabilities, the probability that they'll be revising tomorrow is 30%. But what if they were distracted yesterday?

Still 30%.And what if they have been drafting for the past five days?

Again, 30%.So, essentially, once we know today's state, the model does not use the previous states to determine tomorrow's probabilities.However, an assumption that appears reasonable may not be so in reality. A writer who has been distracted for five consecutive days may behave differently from someone who has been steadily making progress.The Markov property simply assumes that today's state summarizes everything from the past that is relevant for predicting the next state. Whether this assumption is reasonable depends on the system we're modeling and how we've defined our states.We'll return to this question later.3. Understanding the Transition Probability MatrixWe can represent our three-state system using a transition

本文基于 Towards Data Science 的公开内容,由 AI 辅助整理改写后发布。

原标题:How to Build a Markov Chain in Python With NumPy

阅读原文