Why did SpaceX retire Falcon 1? Wouldn't it have been more cost effective then Falcon 9 for lighter missions?
The Falcon 1 was less profitable to maintain, and it didn't have the customer base to support using it. A Falcon 1 launch cost around $10 million, of which about 10% was profit. They also considered a Falcon 1e design, which would carry slightly more to orbit, but only 1100 kg tops.
There were a few people who are known to have booked a Falcon 1 launch, namely DoD (TacSAT-1), MDA Corp, Swedish Space Corp, the US Air Force, and ORBCOMM. Most of these could be done with a Falcon 9, and many were given a huge discount to use the larger rocket.
Keep in mind that the Falcon 1 had a development cost of around \$100 million. At best it would make \$1 million profit per launch. Without considerable demand, it wasn't worth continuing to develop that rocket.
At the same time, NASA came knocking on SpaceX's door with a $400 million contract to build a ISS resupply mission. SpaceX was broke, and it would have been hard to justify continuing the development of Falcon 1 which would never see a huge return on it's money. In the end, they opted to pursue NASA's money, which pretty much required them to abandon the Falcon 1 design entirely.
The bottom line is, Falcon 1 wasn't profitable enough to continue with, and had to be abandoned, in order to make room for the more capable and profitable Falcon 9.
Viewing SpaceX's history, I get the distinct impression that Falcon 1 was a development vehicle: it allowed SpaceX to test the Merlin engine, and it allowed them to demonstrate they were a viable business.
From early on in their history, SpaceX has been trying to lower launch cost through reuse. There were plans to recover the first stage of the Falcon 1 (via parachute landings) but those plans were shelved.
My guess is that the extra hardware needed would reduce F1's payload by too much, or they realized refurbishing a booster after a parachute landing in water wasn't going to be cheap.
SpaceX said the market for Falcon 1-sized payloads was too small to pursue.
Instead, development effort was concentrated on Falcon 9, which has much larger margins for recovery.
SpaceX believes rocket reusability is the key breakthrough needed to reduce the cost of access to space and enable people to live on other planets.
Running 2 programs in parallel (F1/F9) would have spread engineering resources, delaying the F9 program.