A value function tells a machine the value of the current state of affairs - simple as that. In the everyday world, where the machine is the brain, a value function does the same thing--it assigns a number to an outcome, whether or not the outcome is due to external experience or internal experience.