If the filter looks dirty then clean it. I clean mine about every 3 month or so. :)
A tip here: a filter should always smell clean,, if it smells fishy/eggy/just plain nasty youve left it too long before cleaning.
And certainly clean it before going into winter as the bacteria will continue to grow and eventually cause illness in your fish, usually in early spring before the fishes immune system is up and running.
Remember to clean the filter with pond water only.
Wear gloves etc as it has been known when cleaning filters for the cleaner person to get severe illnesses.