The modified Cigarette Evaluation Questionnaire (mCEQ) is a self-report instrument that assesses the reinforcing effects of smoking cigarettes from a smoker’s perspective. Conceptually, the mCEQ consists of three multi-item and two single-item domains. The increasing availability of alternative products to cigarettes raises the question whether the mCEQ can also be used to assess the reinforcing effects of other tobacco and nicotine containing products. The study aimed at a psychometric evaluation of the mCEQ applied to cigarettes and a heat-not-burn tobacco product, the tobacco heating system (THS). Furthermore, the potential to integrate items from two other instruments (Minnesota Withdrawal Scale-Revised, MNWS-R; Questionnaire on smoking urges – brief version, QSUbrief) was investigated. The analysis was based on classical test theory (CTT) and Rasch Measurement Theory (RMT). The data set consisted of a sample collected in two 3-month reduced-exposure studies, one in the U.S. and one in Japan. Within CTT, factor analysis widely confirmed the structure of the mCEQ for both cigarettes and THS. While for two multi-item domains (Smoking Satisfaction and Psychological Reward) the items formed proper scales allowing for exploiting the psychometric benefits of RMT, the third multi-item domain (Aversion) showed extremely poor targeting (resulting in strong floor effects) as a fundamental psychometric problem regardless of the model used to estimate respondent measures. The two single-item domains remain to be interpreted as such, and attempts to extend these to multi-item domains revealed no real potential to improve the mCEQ. In its current form, it is recommended to administer the full mCEQ, which can be applied equally to cigarettes and THS in the U.S. and in Japan. The domains of Smoking Satisfaction and Psychological Reward (after excluding one item) qualify for being analyzed by the Rasch model, while additional items should be added to the domain of Aversion to mitigate the poor targeting.