Inspect the edges of the window casing where it meets the walls and determine how it is held to the wall. Some methods include adhesive caulk, screws and nails.
Slide a putty knife along the edge of the window casing where it meets the wall. This removes any paint that might be holding the casing to the wall. You can also break the caulk seal holding the casing to the wall, if there is one present.
Remove any screws or nails holding the casing to the wall. A screwdriver can be used to remove the screws, while the claw end of a hammer or a pair of pliers pulls up the nails.
Pry the casing away from the window using a pry bar, and set it aside. Work carefully so that you do not bend or break the casing.
Measure the hollow spaces behind the casing using a measuring tape
Cut rectangles of rigid polyisocyanurate insulation to match the measurements you found in in Step 5.
Slide the rectangles of rigid polyisocyanurate insulation into the hollows. If you cut them correctly, they should stay in due to friction. If not, you can use a small amount of double-sided tape to hold them in place.
Reattach the window casing over the window using a hammer and nails or adhesive caulk to put it back in place.