It's natural that a MASH would need a library to store all their medical journals, technical manuals and medical records. It's surprising, however, that the library is so little used and referenced in episodes. It would have, for example, been a good place for a date but the supply tent always seems to be the preferred location for this.
The 4077th MASH library is only shown once, during the Pilot episode. We see Lieutenant "Dish" Schneider looking for some books in the library only to find Hawkeye there waiting to hit on her. There is no information about where the library is but a logical location would be together with the medical supplies in the supply tent.
In "A War for All Seasons" (Season 9) Hawkeye and B.J. debate whether they can build a kidney dialysis machine. Hawkeye tells B.J. to follow him and bring his "library card" in order to read up on the subject.
Lt. Dish in the library. The pole by her right shoulder suggests the structure is a tent of some kind.