I am trying to create a Lunar Flyby mission for small nanosatellites and I want to plot an orbit for it. The problem is that no website has proper information as to what variables, problems and mechanics I need to consider. I have been trying to use GMAT but there are no tutorials for doing the same and the few that are there are too complicated and don't explain what the terms used are. How can I simulate this, in GMAT or other software?
Unfortunately it costs money, but Universe Sandbox may be able to do exactly what you need. I'm not affiliated with it and haven't tried it yet myself, but...
You can take some time and read the several Q&A about it in Astronomy SE: https://astronomy.stackexchange.com/search?q=universe+sandbox
You can also go to Scott Manley's YouTube Channel and enjoy some of his videos where he has fun with Universe Sandbox. Here are a few random screen shots, the software is quite flexible:
Unlike Kerbal Space Program, it does handle many-body problems, so traveling though cis-lunar space feeling the gravity of the Sun, Earth, and Moon should be possible. However as I point out here their excuse for not including General Relativity (e.g. cool visuals near black holes) should not preclude them from including basic relativistic effects in their orbit calculations. Probably not so important for going to the Moon though.
As @RussellBorogove points out in this answer
Relativistic effects didn't have to be modeled; other sources of error would have swamped the effects of relativity, and midcourse corrections were made.
and goes on to talk about the importance of including a lumpy gravity model for orbiting the Moon accurately. That would only be important if you needed to look at orbital perturbations over say tens, hundreds, or thousands of orbits, and so also probably not relevant to your application.