qwen 2.5 coder 32b generates working code but imports dont match actual package structure
been testing qwen 2.5 coder 32b q4_k_m on code generation tasks for the last week and hitting a weird pattern. the model generates syntactically correct code that runs, but the imports are often wrong - imports from submodules that dont exist, or uses the old deprecated import path instead of the current one. for example it generated `from sklearn.cross_validation import train_test_split` which was deprecated in sklearn 0.20 like 5 years ago. the current import is `from sklearn.model_selection import train_test_split`. feels like the training data had a bunch of old stackoverflow answers or something anyone else seeing this? the 7b and 14b models seem fine on imports
tested qwen 2.5 coder 32b q4_k_m yesterday on a flask app codebase and it generated imports like `from flask.utils import get_debug_flag` which doesn't exist - the actual import is `from flask.helpers import get_debug_flag`. the code structure is correct but the package paths are wrong. feels like the model learned patterns but not actual package structure
+1 hit this on react imports too
lol this is because the tokenizer vocab doesn't have common package name subwords. brings it back to my pet topic - tokenizer quality matters way more than people think for code generation
tokenizer quality matters but qwen still hallucinates imports even with perfect vocab
tokenizer quality is huge for code generation but even with perfect vocab qwen 2.5 coder still hallucinates imports that dont exist in the actual package structure. tested qwen 2.5 coder 32b q4_k_m yesterday and it generated `from flask.json import JSONEncoder` which is wrong, actual import is `from flask import json` then `json.JSONEncoder`. the model knows the syntax but gets the package hierarchy wrong