Potential energy is proportional to the object's mass, and to the height
to which it's lifted. If the heights are the same, then it depends only on
the mass, and the 4-kg object has more potential energy at that height.
How long it took each one to arrive at that height is a totally irrelevant
consideration.