Yes, it is. People born in Puerto Rico, Guam, the U.S. Virgin Islands, and the Northern Mariana Islands are recognized by the federal government as “statutory” rather than “Fourteenth Amendment” citizens. Under the federal government’s view - which finds no basis in the text or history of the Citizenship Clause itself (see here) - Congress could not only stop recognizing people born in U.S. territories as citizens moving forward, it could retroactively revoke the citizenship status of people who have been recognized as citizens their entire lives.

A 1996 House Report cited past statement from the U.S. Justice Department to conclude “that the statutory citizenship which Congress has conferred on people born in Puerto Rico during the territory period is not full, equal citizenship protected by the Fourteenth Amendment to the Constitution.” As a result, it bluntly declared that “the current citizenship status of Puerto Ricans exists at the discretion of Congress.” 

Disturbingly, the House Report set forth the view that “the current statutory citizenship of people born in Puerto Rico can be regulated or even rescinded without violating the equal protection and due process rights which have been extended to Puerto Rico by Congress and the Federal courts …  as the fundamental rights test of the Insular Cases . . . is met,” meaning, in its view, “the existence of a legitimate Federal purpose achieved in a manner reasonably related to that purpose.”  The same principles would apply to people born in other territories. 

Read more in Section III of this legal brief filed by Right to Democracy in Trump v. Barbara on behalf of former elected officials and judges from American Samoa and other U.S. territories.