The word "opposite" is found in the definitions of both of these words, so obviously there are similarities. However, they are not quite the same.
"Inverse" comes from the Latin "invertere", meaning to turn upside down or inside out. It is often used in mathematics, as in "3/2 is the inverse of 2/3".
"Converse" comes from the Latin "convertere", meaning to turn around. It is most often used in connection with words, rather than with mathematical concepts. In fact, the misstatement known as a "Spoonerism" is a form of converse, in which letters or syllables are inadvertently interchanged. An example is "lack of pies" rather than "pack of lies", or, the oft-cited one attributed to Rev. Spooner when officiating at weddings, "It is kisstomary to cuss the bride?"