Social perception research demonstrates that people can infer the high-level goals driving many motor actions. But what about the rich visuomotor processes underlying such actions? Visually-guided behavior relies on a complex feedback loop between agents and environments, with subtle corrective adjustments made online. What do observers understand about this dynamic? Here, we explore these questions through “pantomimed actions”. We created a stimulus set of videos where agents performed both genuine object-directed actions (e.g. stepping over a box), and pantomimes of those actions (e.g. stepping over an imagined box). Independent subjects then watched these videos and had to determine which videos were which. Collapsing across actions, observers successfully discriminated real actions from pantomimes. However, certain actions were more discriminable than others. This suggests that (1) observers understand how online visual information shapes human motor behavior; (2) The ability to “fake” actions may be more robust than previously suggested.