In the first problem, books of the same subject should be placed together.
So you can consider books from a particular subject as a single entity.
So instead of M1M2M3M4 and C1C2C3, consider them as just M, C, H and L.
This is done to ensure that books from same subject are always together.
So you have 4 objects (4 subjects) and these can be arranged in 4! ways.
Now suppose one of these arrangement is CHML (Chem, History, Maths, Language).
In this arrangement, the Chemistry block can have 3! permutations (arrangements).
Maths block can have 4! permutations (all maths books are different, if they were same there would be only 1 permutation) and so on.
Similarly for other subject arrangements like MHLC, MLCH and others.
So totally there will be 4!(No of Math permutations * No of Chem permutations * No of History permutations * No of Language permutations)
ie, 4!(4!*3!*2!*1!)
Now consider the case where the books of the same subject are identical, so now you have the books as M, M, M, M, C, C, C, H, H and L.
Books of same subject have to be together, s again consider books of each subject as a single entity. So again we have 4 objects M, C, H
and L. You can arrange these 4 objects in 4! ways. This ensures same subject books are together. But consider any subject block, for eg take the Maths block. There are 4 Maths books but they are all the same. So only one arangement is possible, ie, MMMM. So here there's no need to multiply the answer b 4!*3*2!*1!. So the answer in this cae is just 4!.
Now consider the case where the books of same subject are identical AND there's no restriction on the placement of books (same subject books can be away from each other). We have total 10 books so they can be arranged in 10! ways. But since all books of a subject are same, we can have just 1 arrangement for each subject. But here we have counted many arragements more than once.
Consider these 2 arrangements out of the 10! ways we have counted.
MMMM C1CC3 HH L and MMMM C3CC1 HH L (C1 and C3 are same but they are marked to show difference in their position in the 2 arragements)
They are both included in the 10! ways but only 1 should be included as they are the same.
So to account for this overcounting we have to divide 10! by the number of ways the individual books can be permuted.
So answer is 10!/(4!*3!*2!*1!)
Now try to solve the third problem and ask again if you have some doubts.
Second one is correct.
Check subjectwise weightage here:
http://gatecse.in/mark-distribution-in-gate-cse/