Charlie Nelms was born in 1946 in Crawfordsville, Arkansas. His family members were cotton subsistence farmers and rural community organizers, many of whom moved to Midwestern cities like Chicago and Flint during the post-war second great migration. Nelms earned his B.S. in chemistry and agronomy from University of Arkansas at Pine Bluff in 1968. He then attended Indiana University, where he earned his M.S. in higher education and student affairs in 1971 and his Ed.D. in higher education administration in 1977.

After working in administration at Earlham College and University of Arkansas at Pine Bluff, Nelms worked at Indiana University Northwest as Associate Dean for Academic Affairs and Assistant Professor of Education from 1978-1984. Following a period of employment as Vice President for Student Services at Sinclair Community College (Dayton, Ohio), Nelms returned to the IU regional system as Chancellor and Professor of Education at Indiana University East in 1987. He served in this role from 1987-1994. He left Indiana in 1994 to become Chancellor and Professor of Education at University of Michigan-Flint, a position he served until 1998. Nelms returned to Indiana in 1998 as a member of IU's executive leadership team. He served as Vice President for Institutional Development and Student Affairs and Professor of Higher Education Administration at IU (Bloomington campus) from 1998-2007. In total, Nelms served in higher administration for the Indiana University system for twenty-four years.

In 2007 Nelms became Chancellor at North Carolina Central University (NCCU), a public historically black university. He retired from NCCU in 2012. Since then, Nelms has focused on research and consulting primarily for historically black colleges and universities (HBCUs). Since 2014 he has been a Senior Fellow for the Division of Academic Leadership & Change, American Association of State Colleges & Universities (AASCU) and Senior Consultant for AGB Consulting in Washington, D.C. In 2015 he established Nelms & Associates, a higher education consulting firm that specializes in strategic planning and communications, enrollment management, and diversity and inclusion.

Nelms has written prolifically about education and race throughout his career. In 2011 he published "A Call to Action: Strengthening America's Historically Black Colleges and Universities" after participating in a series of symposiums on the topic at NCCU. He has been a columnist for the Huffington Post since 2012, where he blogs about family, leadership, higher education, and civil rights. His first memoir publication was Lessons Taught, Lessons Learned (Mansfield, Ohio: Bookmasters) in 2004. In 2019, Indiana University Press published Nelms' From Cotton Fields to University Leadership: All Eyes on Charlie, A Memoir. The book details his upbringing in the Deep South, his educational journey, and his experiences as a black administrator in predominantly white universities.

Nelms has been granted honorary degrees from Earlham College (1993), University of Arkansas at Pine Bluff (2001), and Indiana University (2019). He has been awarded The President's Medal from the University of Michigan (1996), the Distinguished Alumni Service Award from Indiana University (1996), the City of Bloomington Dr. Martin Luther King Jr. Legacy Award (2007), the U.S. Department of Education MLK Drum Major for Service Award (2012), and the Distinguished Leadership Award from the American Association of Blacks in Higher Education (2013), among other distinctions.