The FCC voted 3-2 along political lines in a meeting disrupted by shouting.
The rules also call for transparency by Internet service providers and would create an FCC "ombudsman" to investigate Internet complaints.
Thursday's action will kick off months of heated public debate and is the latest attempt by the FCC, dating back a decade, to develop Internet rules guaranteeing that people can get to where they want on the Internet, as well as other protections.
After an initial 120-day public-comment period, the FCC will continue the rule-making process with the hope of having new rules by the end of the year.
A federal appeals court in January voided the prior Internet rules. Verizon, one of the nation's largest wireline and wireless Internet providers, had filed the lawsuit against the FCC to overturn them.
"It has become a political issue because of money and power, because of Silicon Valley and the big elephants of the Internet service providers," said William Baker, the director of the Bernard L. Schwartz Center for Media, Public Policy and Education at Fordham University.
Comcast, which spends millions of dollars a year lobbying in Washington, is the nation's largest residential Internet provider.
"I have a certain sympathy for Tom Wheeler. He is in an impossible political situation," Baker added.
Wheeler, a former industry lobbyist and entrepreneur, said Internet network operators should inform on themselves when they change how they run their networks with "rat-out rules."
Consumers also should be confident that when they buy an Internet service they get the connectivity and bandwidth they paid for, he said.
But critics say Wheeler's proposal isn't as tough as it could be.
Sen. Al Franken (D., Minn.) said Thursday that the FCC proposal could "spell the beginning of the end of the Internet as we know it" because of the possibility of an Internet fast lane, or a company paying to give its data traffic priority, or faster speeds, to subscribers.
More than 36,000 public comments have been filed with the FCC, with some calling for the Internet to be regulated like a phone company, or "common carrier," because of its importance in the lives of everyday Americans.
Wheeler's proposal invites comments on whether the FCC should consider common carrier regulations but does not take this approach.
Republican FCC commissioner Michael O'Rielly was the most critical. He labeled the proposal a "boondoggle" that could result in "monopoly-era telephone rules" stifling the Internet.
Commissioner Ajit Pai, also a Republican, said he supported Internet openness but feared that the agency had "rushed headlong into this rule-making."
The industry was most concerned with the threat of heavier regulation.
"We will continue to reiterate our unwavering opposition to any proposals that attempt to reclassify broadband services under the heavy-handed regulatory yoke" of common carrier provisions, Michael Powell, the head of the National Cable & Telecommunications Association, said in a statement.
Powell, a Republican, is a former FCC head who drafted the first open Internet rules a decade ago.