Auctioneer knowledge of marine product freshness is useful for evaluating and managing the quality of fresh fish. The severity of the working environments in modern fish markets means that it is often difficult to recruit and train new auctioneers in the workplace. Therefore, this study analyzed the physical characteristics employed by auctioneers when assessing fresh fish and described the development of a quality evaluation system for assessing fish freshness. The system analyzed differences in the hue of three different kinds of fish species (Branchiostegus japonicus, Parapristipoma trilineatum, Lagocephalus wheeleri) and compared these against the evaluation criteria employed by auctioneers when assessing meat freshness. Furthermore, appearance evaluation and the relationship this has with the color of the fish surface was undertaken and indexes for color similar to that used by auctioneers was extracted. A model was then designed and used to estimate fish quality by fuzzy inference. The model, designed to use fuzzy inference in combination with the extracted indexes, was applied to both simulated and experimental inspections of fish appearance. Results obtained corresponded with the assessments of auctioneers with a certainty of more than 90%.