Dereham were beaten by an injury-time goal.
After a frantic opening Jamie Forshaw missed a good chance for Dereham after a flowing three-man move.
Forshaw then had a goalbound effort cleared off the line following a corner on the right. As the home pressure continued visiting keeper Lamar Johnson made a fine save from Danny Beaumont's 20-yarder before Rhys Logan steered Beaumont's long ball over from close range.
Gray were awarded a penalty in the 35th minute when Barry Conan was upended, but Koran Bishop's spot-kick hit a post with keeper Elliot Pride making an excellent save from the follow-up.
Grays keeper Johnson denied Beaumont, with the half ending with Robbie Linford making a goal-line clearance to deny Bishop. The dangerous Bishop cut in from the right and clipped Pride's bar in the 88th minute, and Grays stole the points in added time when Adeyinka Cole netted from close range.
Comments: Our rules
We want our comments to be a lively and valuable part of our community - a place where readers can debate and engage with the most important local issues. The ability to comment on our stories is a privilege, not a right, however, and that privilege may be withdrawn if it is abused or misused.
Please report any comments that break our rules.
Read the rules here