While many studies about gray water footprint (GWF) were carried out for different economic sectors, regions, production systems as well as methodologies, little research evaluates improving potential of the current situation. As the largest nitrogen (N) fertilizer consumption country in the world, leaching and runoff contributed half of the N release to waters, while associated GWF was not calculated based on local parameters, management and water quality standards. Different from previous studies, we firstly conducted a meta-analysis of N leaching and runoff parameters based on traditional and optimal management practices from the original field experiments rather than empirical parameters; then, distinguished the different influences of leaching and runoff on groundwater and surface water and applied corresponding standards for the GWF calculations; finally, estimated the improving potential of the GWF. The results show that N leaching and runoff could be reduced by 53%–84% and 34%–68%, respectively, by optimal fertilizer application and irrigation, and integrated management, while the corresponding GWF associated with leaching and runoff could be reduced by 53%–87% and 29%–59%, respectively, without sacrificing crop yields. The total GWF was 999.8 billion cubic meters in 2016, of which 45% was from leaching and 55% from runoff. GWF hotspots were concentrated in the middle and downstream of the Yangtze River basin, while the highest water pollution level (WPL) was found in northern China (e.g., in the Hai River basin, where N released to water exceeded the water resource dilution capacity). It was found that optimal fertilizer and irrigation management could reduce the GWF from existing cropping practices by 44%. We suggest that original experimental results and standards of different water pollution processes should be considered for the GWF calculations, and a GWF assessment under optimal crop management should be used as indicators to guide the technology and policy of water resource management and sustainable intensive agriculture development.