The agency arrived at the 5.5-acre site in late August at the request of the state Department of Environmental Protection.
Officials estimated that the cleanup would likely cost about $1 million, and say they're looking for the owner, Thomas Toy, to foot the bill.
The EPA has been "trying to continue to contact the property owner and work with him, but we have been undertaking several actions to try to encourage him to step forward as well," said Keith Glenn, an EPA on-site coordinator.
Enck did not rule out charges against Toy. Glenn declined to discuss legal matters further.
The EPA is investigating a similar facility in Delaware maintained by Toy.
Attempts to reach Toy and his attorney were unsuccessful.
The Elk site is listed under the EPA's Superfund program, an effort to clean the nation's worst hazardous waste sites.
The facility - used to clean and recondition containers, largely related to the food industry - operated from the 1980s to 2012, when it fell inactive.
The DEP was in contact with site management as early as 2010, overseeing the removal of the growing number of containers after residents complained. The owner stopped cooperating and, according to officials, abandoned the site.
Asked why inspections at the facility had not found the toxic materials, Enck said: "A lot depends on what's in the drums that dictate the level of oversight.
"I don't know if we're going to be able to answer that question. It's a fair one," she added.
DEP spokesman Larry Hajna said tests at the site and of rinse water used to clean the drums did not show the toxic materials.
"Some of the drums were pushed back, not accessible for our investigators to get to," Hajna said.
The EPA used a warrant to access and investigate the site.
Enck noted some of the effects linked to the materials found at the site: Benzene can cause cancer, and lead is a neurotoxin that can result in developmental issues in children.
Officials will continue to monitor the air surrounding the work area and do further samples on soil and groundwater to determine possible contamination.
Preliminary tests indicate concentrations "are not indicative of a groundwater contamination," Glenn said.
But the EPA said more thorough samples would be taken after the containers are gone. The containers will be moved by truck to facilities that accept hazardous material, according to the EPA.