One of my favorite things to do in my classroom is paint my windows. I have a few windows that are open to another room, and I've painted over these. I painted a color wheel on one window and lines on another window (see below). When the lights are on in the other room my window murals glow like stained glass. I used tempera paint, and it easily washes off. High quality tempera paint works the best, especially if you want full coverage of the windows. I choose Blick Student Grade Tempera, but really, any tempera paint will do. Over time the tempera paint may start to flake off, and children are often tempted to scratch at it, but my color wheel and line chart have been up for over a year and I've only had to do a few touch ups.
If you have a window on the door of your classroom you can start here! I enjoy painting my door window with seasonal decorations, I change it up every month or two. Currently it is leaves for Fall. Kids are very tempted to scratch at these, but it doesn't bother me, because I do switch them out quite often. And they are super easy to wash off with a big sponge and water.
Always paint on the inside of the windows. If you paint on the outside it will get moist and flake off much faster. Look around your room, where can you paint windows? Let me know if you come up with some good ones!