When we add a server to our list in the Minecraft launcher we can see that a square image appears to its left, it is its server icon. If you still do not know how to put this image, read on.
For this you need a file manager like Filezilla to connect to FTP. Also, some hosting have an online file manager. You just have to add the PNG file in the main folder of the server and rename it to "server-icon.png"