The Japanese H-IIB seems like an interesting cost effective system. I see that it can carry a hefty payload into Earth orbit. Could it send a smaller payload to the Moon?
Almost certainly. The predecessor H-IIA rocket delivered the orbiter SELENE to the moon in 2007. The SELENE-2 mission slated for 2018 or 2019 will use either an H-IIA or an H-IIB launcher -- it's not entirely clear which from the references I've found, but I think it will be the H-IIA, as the payload is not significantly heavier than that of SELENE.