A new five-storey building costing £30m will include hi-tech exhibitions, offices and teaching space.
A petition with more than 2,500 names objected to the loss of open space, but backers promised new jobs and visitors.
The site was previously home to the now-dismantled Tube visitor centre from 1993 to 2010.
The museum’s proposed move to Cardiff Bay was announced by the UK government as part of the chancellor’s Autumn Statement in 2016, albeit with a different site in mind.
Planning agents for the museum said its present location at Mychett, near Aldershot, was not very accessible to the public, and the move to Cardiff “would allow the armed forces medical story to be told to a far greater audience”.
The museum’s collection of nearly 20,000 objects tells the story of how war inspired medical advancements such as X-rays, trauma and burns care.
The new premises also hopes to offer a super-high definition 8K immersive video space, showcasing new medical and scientific innovations.
Councillor Rodney Berman, who opposed the proposal, told the planning committee the council had previously stepped in to buy the land in 2018 to prevent a 24-storey apartment block from being built there.
“There are plenty of alternative sites where the museum could go elsewhere in Cardiff Bay,” he said.
Also speaking against, Councillor Iona Gordon said the area was already lacking open space for use by a growing population.
Speaking in favour, Councillor Mike Jones-Pritchard said the bay had hard open spaces such as Roald Dahl Plass but few visitor attractions.
“This facility here will provide a pull to the public. Buildings pull people to places,” he said.
Councillors on the planning committee were split, with six voting in favour, three against, and one abstaining, according to the Local Democracy Reporting Service.
The children’s play park and Grade II-listed Lockkeeper’s Cottage at the site will be relocated.
The museum’s director, Jason Semmens, hailed the decision as “a major milestone in our vision to create a world-class visitor attraction”.