Amateur liquid rocketry is doable but it is expensive. There are only two oxidizers for the amateur, liquid oxygen and nitrous oxide. An alternative is hydrogen peroxide. It is doable but you may need to distill it from 30-50% to 80-90%. Fuels can be kerosene, propane, and alcohol.
To build a system is a major effort. Check out the numerous teams competing for the Base 11 Space Challenge. Many teams are spending money but they all have a long way to go.
My advice is to start small, 50 to 200 lbf of thrust for the first engine. Start with a run time of 10 seconds. Short runs mean small tanks. Be careful of material compatibility. Do lots of research to determine what sort of equipment you will need. You will need access to machine tools (mill, lathe, welder, grinder, etc.) to make custom pieces.
Finally, it takes lots of time. It is nice to have a group of people involved.