Can VLF be used to create a Mars barrier?
The short answer is no because Mars does not have an intrinsic magnetic field, thus no uniform dipole magnetic field to generate a magnetosphere. I wrote an answer about Jupiter that may be of interest as well at https://physics.stackexchange.com/a/142922/59023.
In the Earth's magnetosphere, the VLF frequency range corresponds to an electromagnetic wave mode known as a whistler wave. These come in a lot of different flavors (e.g., chorus, hiss, lightning-generated, lion roars, etc.). How they form is not fully resolved, but we have a decent idea and it is as follows.
During a substorm (or some other geomagnetic activity), electrons and ions are brought earthward from the geomagnetic tail. During this transit, the particles are pushed into regions of larger magnetic field magnitude (and the topology is more dipole-like than deep in the tail). Some particles flow directly along the magnetic field and are lost to Earth's atmosphere while others are reflected by the magnetic field gradient (i.e., it's like a magnetic mirror). So what researchers observe when they examine particle velocity distribution functions (VDFs) is an oblate distribution with the perpendicular (to the quasi-static magnetic field) direction being larger than the parallel. The distortion/oblateness results from the loss of particles to the atmosphere. The angular region in phase space where particles are lost versus those reflected by mirror forces is called the loss cone.
Anyways, such a VDF is unstable to something called a temperature anisotropy instability. Basically the particles have too much random kinetic energy perpendicular to the magnetic field, so some of the thermal oscillations of the plasma absorb that free energy and grow in amplitude. The modes that gain the energy are on the whistler mode branch of the dispersion relation.
Note that while the whistlers are being radiated, we must still conserve energy/momentum. So the particles in the VDF lose random perpendicular kinetic energy and are scattered as well. The end result is a more isotropic VDF. Now the whistler waves propagate away from their source regions and act on other VDFs. If the whistler wave propagates at an oblique angle to the magnetic field, it will be able to interact with charged particles through two mechanisms: cyclotron and Landau interactions (see https://physics.stackexchange.com/a/630629/59023 for more details).
I just ran across some older articles in 2017 discussing how VLF radio waves created an artificial barrier to space weather. Basically how VLF improves Earth’s protection from solar radiation.
These waves can lose amplitude and give energy to particles, usually electrons, which then acts to "clear out" regions of space around Earth, but only in the energy bands resonant with the waves. That is, this "barrier" is really poorly named and really only applies to PR-related stuff. It's only a "barrier" for a specific range of energies.
This has really nothing to do with solar radiation either. The radiation belts are populated, mostly, by energizing electrons that came from the geomagnetic tail. It is true that many of these were once of solar origin, but not all. Regardless, the particles don't come directly from the sun and then hop into the radiation belts.
Well I should be careful. There are two primary radiation belts, an inner and outer. They are nearly toroidal in shape, though their cross-sections look more like croissants than circles. The inner belt is almost exclusively protons and the outer mostly electrons. The particles in the inner belt have remarkably long lifetimes, on the order of years to many decades. The electrons in the outer belt, however, can be lost in seconds by hitting the magnetopause (called magnetopause shadowing) or to Earth's atmosphere (called particle precipitation) or they can remain relatively stable for several days.
Side Note: If you look up a Project Starfish you will discover we humans were rather foolish and ignited a nuclear weapon in the upper ionosphere. Some of the heavy ions generated by that explosion are still bouncing around in the inner of the two main radiation belt tori.
This made me wonder if the use of VLF would have similar effects on Mars to increase protection from solar radiation? Would such levels of protection be of any significance?
No, and basically none. As I said above, Mars does not have a uniform magnetosphere or stable radiation belts. The "barrier" that was made by Navy VLF transmitters only worked for very energetic electrons (if memory serves me, they were >1 MeV), which is at the very tail of the VDF (i.e., the lowest abundances).