There is a specific genetic difference in how caffeine is cleaned up in people. Some people are rate limited by the ambirnt CYP1A2 in their system, and others produce more to meet the demand in response to caffeine.

For the caffeine sensitive there is like a 5 hour half life for caffeine in their system.