First, they asked about False option
Option-A : True
When the rank[A] = rank[A,B] then only the system of linear equations has the solution
If rank[A] is not equal to rank[A,B] then system of linear has No solution
Option-B: True
They have mentioned that b is a zero vector. So [000…0|b] this case is not possible. I.e No solution case is not possible
When m<n then at least one column will be free . So if there is at least one free column then the system of linear equation has infinite solutions
Option-C: False
Given b is a zero vector so [000…0|b] this case is not possible.i.e, No solution case is not possible
They mention m=n but they did not mention the rank.
If the rank[A|b] = m then only the system has unique solution.
If the rank[A|b] < m then the system can have infinite solution Because there will be a free variable
Option-D: True
Given m=n and b is a zero vector so the equation is AX=0.
Now they mentioned that rank[A] = n. That is every column is linearly independent.
Important point here is all the columns are linearly independent so there will be no non-zero vector X which makes AX=0. The vector must be zero vector (trivial) which results AX=0