Here is a mathematical answer to your question. Using the numbers you provided for the HG-3, we have,
$$I_{sp} = 451 \ \text{s} \ (\text{vac})$$
$$F = 1,400,745 \ \text{N} \ \ (314,900 \ \text{lbf}) \ (\text{vac})$$
From this alone, we can get an estimate of the steady mass flow rate through the engine,
$$ \dot{m} = \frac{F}{I_{sp} g} = 317 \ \text{kg/s}$$
Now we can figure out the size of the throat from the choked flow condition at the nozzle throat. The mass flow parameter can be rearranged to the form,
$$ A_t = \frac{\dot{m} \sqrt{T_1}} {p_1} \sqrt{\frac{R}{\gamma}} \left(\frac{\gamma+1}{2}\right)^{\frac{(\gamma+1)}{2(\gamma-1)}}$$
where $\dot{m}$ is the flow rate through the engine, $T_1$ is the chamber temperature, $p_1$ is the chamber pressure, $R$ is the specific gas constant of the burned products, and $\gamma$ is the ratio of specific heats of the burned products. Using the NASA Chemical Equilibrium with Applications (CEA) code for an average LOX:fuel mixture ratio of 5.2:1 at a chamber pressure of 3,000 psia, we obtain,
$$ T_1 = 3,414 \ \text{K} $$
$$ R = 678 \ \text{J/kg$\cdot$K} $$
$$ \gamma = 1.16 $$
Substituting these values with the previously calculated mass flow rate yields,
$$ A_t = A^{*} = 0.0363 \ \text{m$^2$} \ \ (0.3910 \ \text{ft$^2$})$$
$$ d_t = d^{*} = 0.215 \ \text{m} \ \ (0.705 \ \text{ft})$$
Now from your provided information, we were given that $d_e$ = 2.03 m, namely, $A_e$ = 3.235 m$^2$. So our nozzle expansion area ratio required to obtain a vacuum $I_{sp} = 451 \ \text{s}$ is given by,
$$ \boxed{ \epsilon = \frac{A_e}{A_t} = \frac{A_e}{A^*} = 89.1 }$$
That is the answer to your first question.
Now we want to look at the sea-level performance. We will assume the mass flow rate through the engine is similar to the vacuum scenario, since the LOX/fuel mixture ratio only fluctuates by $\pm$ 1 and we already took the average for sizing the nozzle throat above. We need to determine the nozzle exit parameters. For idealized quasi one-dimensional compressible flow, the Area-Mach relation can be used to estimate the nozzle exit Mach number.
$$\frac{A_e}{A_t} = \frac{A_e}{A^*} = \frac{1}{M_e}\left[\frac{2}{\gamma+1} \frac{\gamma-1}{2}M_e^2\right]^{\frac{\gamma+1}{2(\gamma-1)}}$$
Using the Newton-Raphson numerical technique to solve the above for $M_e$ given $A_e/A_t$ = 89.1 we obtain,
$$ M_e = 4.52 $$
It is conventional in idealized rocket analysis to assume isentropic expansion of the burned products through the nozzle. Hence, from isentropic relations we have at the nozzle exit plane,
$$ p_e = p_1 \left(1 + \frac{\gamma-1}{2} M_e^2 \right)^{-\frac{\gamma}{\gamma-1}} $$
$$ T_e = T_1 \left(1 + \frac{\gamma-1}{2} M_e^2 \right)^{-1} $$
To which we obtain,
$$p_e = 18,735 \ \text{Pa} \ \ (2.72 \ \text{psia}) $$
$$T_e = 1,285 \ \text{K} \ \ (1,853 \ \text{$^{\circ}$F}) $$
Further, the sound speed on the nozzle exit plane is given by,
$$a_e = \sqrt{\gamma R T_e} $$
To which we obtain,
$$ a_e = 1,006.5 \ \text{m/s} \ \ (3,302 \ \text{ft/s}) $$
Hence, our nozzle exit velocity is simply,
$$ V_e = a_e M_e = 4,551 \ \text{m/s} \ \ (14,931 \ \text{ft/s}) $$
Now our sea-level thrust is given by,
$$ F = \dot{m} V_e + \left(p_e - p_a\right) A_e $$
where at sea-level, $p_a$ = 101,325 Pa (14.7 psia). Substituting all of the calculated parameters yields,
$$ F = 1,173,595 \ \text{N} \ \ (263,834 \ \text{lbf} ) $$
Similarly, the sea-level specific impulse is given by,
$$ I_{sp} = 378 \ \text{s} $$
This is obviously based on idealized rocket performance analysis. Additionally, the sea-level specific impulse is very large compared to the 280 you presented in the question. So what is going on here? Well, it turns out we are going to suffer performance losses from over-expansion with a nozzle that has a nozzle expansion area ratio of roughly 89. According to Sutton (Rocket Propulsion Elements), if $p_a$ is slightly higher than $p_e$ then we will have our idealized $I_{sp} \approx$ 378 and our nozzle will continue to flow full. He further states that the nozzle will continue to flow full while developing a slight contraction until the nozzle exit pressure drops to about 40% of the ambient. However, in our case we have very severe over-expansion at the sea-level condition, where our nozzle exit pressure is only approximately 18% of the sea-level ambient back pressure condition. According to the Summerfield criterion (Flow Separation in Over-expanded Supersonic Exhaust Nozzles”, Jet propulsion, Vol. 24, No. 9, page 319-321, 1954), which is a general rule of thumb, the nozzle flow will separate at only $p_e/p_a$ = 0.4. So in our case, we will definitely have flow separation in the divergent portion of the nozzle, which will drastically undercut our anticipated ideal $I_{sp}$. Below is a schematic from Sutton's text that demonstrates the effects of flow separation in the diverging portion of the nozzle.
The general answer to your questions go as follows:
1.) Yes the HG-3 engine could achieve a vacuum $I_{sp} = 451 \ \text{s}$ provided the nozzle expansion area ratio was roughly $\epsilon = 89$.
2.) Nozzle expansion area ratio plays a large role in the over-expansion effects regarding the nozzle flow losses and separation. Cutting back the expansion area ratio to something similar to the SSME (69) alleviates this issue at the sea-level condition.
3.) The sensitivity of sea-level performance to over-expansion is much more severe than the performance gains from higher expansion area-ratios in a vacuum back pressure condition. From the NASA CEA code and the conditions previously mentioned, the vacuum specific impulse goes as follows:
$$ \begin{array}{c|c}
\epsilon = \frac{A_e}{A_t} & I_{sp} - \text{s} \ (\text{vac}) \\
\hline
60 & 445 \\
70 & 447 \\
80 & 449 \\
90 & 451 \\
100 & 452 \end{array} $$
I could also tabulate the sea-level performance, however, it is also the ideal sea-level performance and won't account for the performance losses due to over-expansion effects. Determining the losses due to over-expansion effects requires either full numerical solutions to the Navier-Stokes equations, or experimental testing.
