JEE Main · 2024 · 01 Feb 2024 · Shift 1 · Circle

$C:x^2+y^2=4$, centre $(0,0)$, $r_1=2$. $C^{\prime }:(x-2\lambda {)}^2+y^2=4{\lambda }^2-9$, centre $(2\lambda ,0)$, $r_2=\sqrt{4{\lambda }^2-9}$ (real only for $|\lambda |\ge \frac{3}{2}$). Distance between centres $d=2|\lambda |$.

For $|\lambda |<\frac{3}{2}$, $C^{\prime }$ is not a real circle, so no real intersection occurs there. For $\lambda =t\ge \frac{3}{2}$, the two circles meet at two distinct points iff $|r_1-r_2|<d<r_1+r_2$:

*Upper bound* $2t<2+r_2\Rightarrow r_2>2t-2$. Since $2t-2\ge 1>0$, squaring: $4t^2-9>4t^2-8t+4\Rightarrow 8t>13\Rightarrow t>\frac{13}{8}$.

*Lower bound* $|2-r_2|<2t$: since $r_2<2+2t$ always holds here (as shown by squaring, $4t^2-9<4t^2+8t+4$, true for $t>0$) and $r_2>2-2t$ holds automatically since $2-2t<0\le r_2$, this bound is never violated.

So for $t\ge \frac{3}{2}$, intersection at two points holds only for $t>\frac{13}{8}$; by the symmetry $\lambda \to -\lambda$ (both $d$ and $r_2$ depend only on ${\lambda }^2$), for $\lambda \le -\frac{3}{2}$ it holds only for $\lambda <-\frac{13}{8}$.

So the set of $\lambda$ giving two distinct intersection points is $(-\infty ,-\frac{13}{8})\cup (\frac{13}{8},\infty )=\mathbf{R}-\left[-\frac{13}{8},\frac{13}{8}\right]$, giving $a=-\frac{13}{8},b=\frac{13}{8}$.

Testing $(-1,6)$ in option (4): $6(-1{)}^2+6^2=6+36=42$ — satisfied.