In 1948, Harold Lancour, then associate dean at the University of Illinois library school, first informally proposed the idea of a library science honor society. Thirteen distinguished faculty members in the field of library science selected the first students to receive invitations. With those students, the Beta Phi Mu Library Science Honor Society was officially founded in August 1948, with the purpose of recognizing superior academic achievement in library and information science and for sponsoring professional and scholarly activities. In 1969, Beta Phi Mu was formally admitted to membership in the Association of College Honor Societies, and in 1997, Beta Phi Mu became affiliated with the American Library Association. The motto adopted by the society reflects the service orientated nature of library science careers: Aliis inserviendo consumor or "Consumed in the service of others." The insignia, a dolphin and an anchor, replicates the mark of the Venetian printer, Aldus Manutius, founder of Aldine Press.

The Chi Chapter of Beta Phi Mu at Indiana University was established in 1971. The chapter sponsors career development workshops for the Library and Information Science students at Indiana University-Bloomington, offers several student scholarships, and recognizes outstanding achievement in the field of library science. They also hold yearly initiation ceremonies to induct new members into the local and international chapters of Beta Phi Mu.

There are three ways to be invited to join the honor society: completion of all requirements leading to a master's degree with a scholastic average of no lower than 3.75 where A equals 4.0, or completion of planned programs of advanced study beyond the master's degree which require full time study for one or more academic years with a scholastic average of 3.75 where A equals 4.0, or recommendation for membership by the faculty of the school attesting to the student's professional promise.