End to end encryption means that a group of people agree to establish encrypted communication with each other using a method that does not allow the servers negotiating that communication to decrypt the contents of the communication. There is sometimes a mechanism to allow the people to identify each other out of band to verify trust. In some cases asymmetric encryption is used. In simpler implementations, symmetric encryption is used with a preshared key.
An example of asymmetric encryption would be OTR (off the record), which has been used in the past to encrypt message payloads in popular chat programs. In those cases, the chat programs are entirely unaware of OTR, or may just see it as a plugin. OTR will encrypt messages with the keys of the party members for whom the messages were intended. Implemented correctly, the servers handling the transport of the communication will never see any private keys nor would "backdoors" be feasible. A chat party member would have to be compromised to glean any discernible data. This gives the party members privacy and the server operators plausible deniability about any communications.
End to end encryption of course also assumes the chat servers have no control over the applications and can not push an update the a specific person or group of people and could not mitigate or simply back-door the implementation of E2E. So for example, if I connect to an IRC server and use OTR, the most the server could do is block me for sending words that do not look like unencrypted text of a known language. It can't hijack my application. On the other hand, cell phone applications can be updated by the carrier or authorized maintainers of the application.