BALTIMORE (WJZ) — More than 100 people rallied Wednesday in downtown Baltimore against the Supreme Court’s ruling to uphold President Donald Trump’s travel ban.

“Hey hey, ho ho, Muslim ban has got to go,” protesters chanted at Baltimore City Hall.

Muslims in Maryland say they live in fear of traveling to restricted countries — not knowing if they’ll be allowed to come back to the U.S. The ruling could directly impact 350,000 Marylanders.

The long-awaited gavel dropped Tuesday, delivering a blow to immigration activists.

“They’re sensing the hostility, the animosity from their neighbors and their co-workers and people in the community. So, there’s a very real sense of fear of what the implications of this ban are,” said CAIR Director of Maryland Outreach Zainab Chaudry. “Right now, we are literally in a battle for the soul of America.”

The Council on American-Islamic Relations organized the rally, but messages of solidarity from other groups were also on prominent display.

“We know that the same administration that are continuing to attack our Muslim brothers and sisters are attacking our immigrant communities,” said Lydia Walther-Rodriguez with CASA of Maryland.

By a vote of 5-4, the Supreme Court ruled that Trump’s travel ban would stand, upholding restrictions in place since December.

The ban largely locked the borders to people from five countries with mostly Muslim populations: Iran, Libya, Somalia, Syria and Yemen. It also affects some travelers in North Korea and Venezuela.

Student Aban Jones said the ban prevented his grandmother from coming to College Park, Maryland, to watch him graduate high school earlier this year.

“You want your family to be there with you when you succeed, so it was definitely tough,” he said.

The nation’s highest court said the policy is “squarely within the scope of presidential authority.”

After the ruling, Trump tweeted:

Republican political leaders showed their support.

“This was the right decision. Federal immigration law specifically gives the president the authority to make determinations in the national interest to keep us safe,” said Sen. Ted Cruz (R-Texas).

Democrats did the opposite.

“It’s un-American to do what has been done,” said “It’s un-American to do what has been done,” said Senate Democratic Leader Charles Schumer.

Chaudry called it state-sanctioned discrimination, with the fear that other presidential policies could piggyback off the controversial decision.

“It just strengthens our determination to continue to fight back,” she said.

Trump has said the policy is in place to keep Islamic extremists out. The order created chaos in airports across the country when it went into place with little warning late last year.

WJZ’s Kimberly Eiten and Ava-joye Burnett contributed to this story.

Follow @WJZ on Twitter and like WJZ-TV | CBS Baltimore on Facebook