In the sermon, the connotation was that man (and wo-man) was content in the garden under the trust that God would provide for their needs. A state of "blissful ignorance", insulated from the world, so-to speak. It was the serpent who, under the guise of knowledge, tricked Eve into taking the fruit. In this sense, gaining knowledge was what expelled Adam and Eve from Eden and lost [them] childhood ignorance.
"Ignorance" as the pejorative we use was a product of early industrial period progressivism. The term came to use to describe that which you should be aware of, but were not, thus implying a deficiency of competency. The famous reference to "Ignorance and Want" from Charles Dickens', A Christmas Carol, is an illustration of "ignorance" as an undesirable trait, specifically the undesirable trait of being ignorant of the plight of fellow man. The Latin, ignorare, was a reference to ignore, or not pay attention to. The common usage would have been applied to a topic about which an individual would have no need to be knowledgeable. A farmer may be ignorant of court custom, a banker may be ignorant of fabric dying, etc. Many biblical references, especially KJV and older, apply ignorance in this sense. Some New Testament references clarify ignorance with "willful" to apply the perjorative deriliction of responsibility. Any newer books than KJV and this all goes out the window, of course.
Ultimately, judging is a by-product of knowledge and something against we must be vigilant. So in that sense, Adam and Eve compared Eden to a utopian imagery crafted by the snake and were fooled in to judging Eden to be insufficient. It follows that one of the first things Adam and Eve realize is their nakedness in the wildreness, their inability to protect even their bodies from the world. Ouch.