Fireworks need three things to operate, heat, fuel and oxygen*. Luckily for the idea of fireworks on the Moon or Mars, many fireworks use a pyrotechnic composition that supplies their own oxygen which is why the devices will work under water or in an inert environment. Any fireworks operating in space, on the Moon or on Mars would need some slight changes from earth fireworks, like adding more oxidizers and adding an electric match for ignition. I am listing a few different effects that already operate in space to show how not only is the idea plausible, we're already doing something similar today.
Rockets

A good example of a firework operating in space is the rocket. Rockets in the fireworks world use a motor made from solid-propellant with a combination of black powder, an oxidizer with metal powders like titanium, iron or other metallic powders to create the sparks leaving the rocket. Pyrotechnicians call the effect a tail. The above photo was taken at fireworks festival called Western Winterblast in 2014 when I launched 2500 rockets at once. The closest example in the rocketry world would be solid fuel rockets like the Space Shuttle Solid Rocket Booster, which are fueled with ammonium perchlorate (oxidizer) and atomized aluminum powder. The separation between the Shuttle and the boosters happens at around 146,000 ft and they rise to around 220,000 ft, which means the rockets operate in space.
As beautiful as a giant tail would look coming from any rocket, adding a metallic powder to the fuel would add weight and decrease thrust and a number of other problems, which I am not prepared to list.

The same effect could be done with liquid rockets. The photo above by Mike Woolson is from a fireworks show at Burning Man 2009 where a group called Black Rock FX staged a simulated launch of a retro-styled rocket built by a group known as Five Ton Crane. The effect of a translucent fireball is called a ghost mine, which consists of methanol, a lifting charge, with metallic powders or chemicals added to color the flame. The problem is that fuels like RP1 are hard to color because of the presence of sodium in the fuel would create yellow flames, which we have discovered overpowers any other colorant. Reds, greens and blues become much harder to produce.
Mines

Another firework already in space is the mine. In the fireworks world, a mine is a ground effect with a lifting charge designed to lift a pyrotechnic composition into the air. Usually the composition is called a star, which are a composition of black powder, an oxidizer, chemicals for colors or metals. The photo was taken at Western Winterblast and features mines from Spirit of 76 fireworks. In space, the same concept is used to separate objects like a booster rocket from the main rocket in a directed explosion. The difference is the lack of chemicals and metals to add color to the detonation. However, the effect could be repurposed to create a color display.
Comets
Another effect fired from a mortar tube is called a comet. It is a rising effect, which means it does not have that classic burst in the sky. Instead, a lifting charge ignites it creates a tail of sparks into the air. Sometimes these are attached to a mortar shell which breaks in the sky. The best example is the shell fired from a flare gun. These could definitely be engineered to be fired on the Moon or Mars and create tails of color into the sky.
Artificial Meteors

Fireworks in space is not a radical concept. One of the reasons it has not been done is we have a limited amount of visits to the Moon and none to Mars. However, fireworks in space is an idea that was presented for the 2020 Tokyo Olympics. A Japanese company ALE (Astro Live Experiences - see photo above) was planning on painting the sky above the Tokyo Olympics with artificial meteors for the Opening Ceremonies. Due to the Olympics being postponed, this may not happen until 2023. The entire concept is using small metal balls to create different colors as they enter the atmosphere over the event.
If any of you are planning on launching a cubesat into space, I hope you can find a way to add a small payload to do your own pyrotechnics in space.
Good luck.
- I mentioned you need three things to create fire, but technically there's four. You can also produce fire through chemical reaction. A good example would be hydrazine fuel.