This is how the sand simulation in Sand Flag works. It's pretty simple!

Internally, the sand is a grid of points with height values. To visualize it, there is a mesh with a sand texture that is shaped according to the height values.

When the Player touches a point on the sand, the point loses its height. Then the point becomes "unstable" for the next 0.5 seconds.

While the instability lasts, the unstable point steals heights from neighboring points that are higher. Then the neighboring points inherit the instability with reduced duration.

The process repeats every 0.1 seconds until the inherited instability becomes less than a certain threshold.

That's it! 

At first, I tried averaging the heights of nearby points so it's like using a smoothing tool in Maya or Blender. But it felt unnatural. This simple way of addition and subtraction of heights ended up looking better.