These are harder to diagnose because the background spectra you see will depend on the contaminants. If the contamination problem is coming from the GC, but it is not septum or column bleed, try:
Cleaning the injection port
Replacing the injection port liner
Cleaning or replacing the syringe
Switching to a different source of carrier gas and/or replacing the gas traps
Replacing the carrier gas tubing with new (clean) tubing
The Agilent Technologies application note: Optimizing splitless injections on your GC for high performance MS analysis