Simple answer - because OS or programmer knows nothing about physical address. It is handled by MMU. Whatever address is being used by the programmer is the same being used by CPU (when CPU executes corresponding instruction) and as given in text books, this is virtual address.
When a program runs, a chunk of Virtual memory is given to it. And you are right- size of a pointer is the no, of bits needed for addressing virtual memory.
I did not get what is meant by operating system needing unique address?