In other words, how big should an entomopter be so that it could fly in Mars' atmosphere and carry a payload of 1-2 kg? I'm trying to find out if an entomopter could be used to carry a spectral sensor camera (special kind of spectral sensor that would be smaller than those already existing). I'm also considering a robotic ornithopter for that mission. Do you know any other kinds of MAV's or UAV's suitable for carrying spectral sensors?
As Hobbes mentioned, there was study funded by the seed program NASA Institute for Advanced Concepts in 2002 to look into the viability of entomopters for Mars exploration.
The results are available online. The team relied mainly on design analysis, but appears to have built a working version of the vehicle's "reciprocating chemical muscle" (RCM) for a small prototype.
The study's primary design point on page 49 suggests an entomopter with a 1.2m wingspan could have a relative lifting capacity of 1.5kg on Mars. Later, the study clarifies that only some of this mass is available as payload:
The weight of the RCM system plus a 10 minute fuel charge is on the order of 0.42 kg. Based on Table 3-2, this would leave 1.08 kg for mission payload equipment.
The RCM system in the proposed entomopter uses fuel, not electricity, meaning a user looking for more than 10 minutes of flight would have to sacrifice even more payload mass.
The study itself suggests using in-situ resource production to help overcome this, but this is itself another unproven technology. Even returning to a refueling station containing stockpiled fuel seems likely to be difficult - the proposed vehicle is not capable of hovering, making a precision landing unlikely.
In conclusion: it seems plausible that an entomopter could one day lift a payload on the scale you describe on Mars, but the vehicle does not seem like a great fit for the current model of slow, long term robotic exploration. To my knowledge, NASA has not funded a follow-up study for the concept.
This study suggests that an entomopter with reasonable wingspan (on the order of 1 m) could achieve a 1 kg payload, if powered by a highly efficient engine running on e.g. hydrazine.
Based on the research conducted during this NIAC Phase II study, a design space has been identified in which Mars Entomopter flight is practical.