★ FEATURED ARTICLE 一、Qwen3-VL-Embedding 模型Qwen3-VL-Embedding系列模型是通义千问家族的最新成员基于强大的Qwen3-VL基础模型构建而成与此同时开源的还有Qwen3-VL-Reranker系列。该系列专为多模态信息检索与跨模态理解而设计可接受包括文本、图像、截图和视频在内的多种输入形式以及这些模态任意组合的混合输入。Qwen3-VL-EmbeddingModelScope地址https://modelscope.cn/models/Qwen/Qwen3-VL-Embedding-8B本文基于Qwen3-VL-Embedding-8B模型本地化部署并实现文搜图案例效果如下所示实验所使用的主要依赖版本如下transformers4.57.0 qwen-vl-utils0.0.14torch2.8.0二、Qwen3-VL-Embedding-8B 部署下载模型modelscope download--modelQwen/Qwen3-VL-Embedding-8B--local_dirQwen3-VL-Embedding-8B然后将下载后scripts和下面的服务代码放在一起读取模型并启动api服务importtimefromfastapiimportFastAPI,Requestfromfastapi.middleware.corsimportCORSMiddlewareimportuvicorn,jsonfromscripts.qwen3_vl_embeddingimportQwen3VLEmbedderimporttorchimportbase64fromioimportBytesIOfromPILimportImage appFastAPI()app.add_middleware(CORSMiddleware,allow_origins[*],allow_credentialsTrue,allow_methods[*],allow_headers[*],)## 下载模型位置model_name_or_path/workspace/modelscope/Qwen3-VL-Embedding-8BmodelQwen3VLEmbedder(model_name_or_pathmodel_name_or_path)defbase64_to_image(image_base64:str):image_database64.b64decode(image_base64)image_fileBytesIO(image_data)imageImage.open(image_file)returnimagedefget_features(text,image):特征提取input{}iftext:input[text]textifimage:input[image]imagereturnmodel.process([input])[0].tolist()app.post(/v1/embeddings)asyncdefembeddings(request:Request):globalmodel json_post_rawawaitrequest.json()json_postjson.dumps(json_post_raw)messagesjson.loads(json_post)textmessages.get(text)imagemessages.get(image)ifnottextandnotimage:return{code:400,message:texts 和 images 至少传一个,}ttime.time()ifimage:imagebase64_to_image(image)embedsget_features(text,image)use_timetime.time()-tiftorch.backends.mps.is_available():torch.mps.empty_cache()return{code:200,message:success,data:embeds,use_time:use_time}if__name____main__:uvicorn.run(app,host0.0.0.0,port8848,workers1)启动后大概占用31.4G显存。三、API 调用示例importbase64importrequestsdefencode_image(image_path):withopen(image_path,rb)asimage_file:returnbase64.b64encode(image_file.read()).decode(utf-8)defembed(textNone,imageNone):ifnottextandnotimage:raiseException(embed content is empty)ifimage:imageencode_image(image)responserequests.post(http://127.0.0.1:8848/v1/embeddings,json{text:text,image:image})returnresponse.json()[data]defmain():texts你好小毕超imagesimages/dog.3999.jpgemembed(texttexts,imageimages)print(f维度:{len(em)})print(em)if__name____main__:main()调用结果向量维度为4096维。四、实现 文搜图 案例4.1 实验图片收集这里我随便从网上下载了一些 猫、狗的图片4.2 Embedding 存储至向量库通过Qwen3-VL-Embedding模型向量化并持久化到Milvus向量库中 。importjsonimportosimportbase64importrequestsfrompymilvusimportMilvusClient,DataType clientMilvusClient(http://127.0.0.1:19530)collection_nameqwen3_vl_testdefcreate_collection():client.drop_collection(collection_namecollection_name)schemaMilvusClient.create_schema(auto_idFalse,enable_dynamic_fieldFalse,)schema.add_field(field_nameid,datatypeDataType.VARCHAR,is_primaryTrue,max_length255)schema.add_field(field_namevector,datatypeDataType.FLOAT_VECTOR,dim4096)schema.add_field(field_namecontent,datatypeDataType.VARCHAR,max_length5000)schema.verify()index_paramsclient.prepare_index_params()index_params.add_index(field_namevector,index_typeIVF_FLAT,metric_typeL2,params{nlist:1024})# 创建 collectionclient.create_collection(collection_namecollection_name,schemaschema,index_paramsindex_params)defencode_image(image_path):withopen(image_path,rb)asimage_file:returnbase64.b64encode(image_file.read()).decode(utf-8)defembed(textNone,imageNone):ifnottextandnotimage:raiseException(embed content is empty)ifimage:imageencode_image(image)responserequests.post(http://127.0.0.1:8848/v1/embeddings,json{text:text,image:image})returnresponse.json()[data]defto_milvus():forindex,imginenumerate(os.listdir(images)):img_pathos.path.join(images,img)emembed(imageimg_path)client.upsert(collection_namecollection_name,data{id:str(index),vector:em,content:img})print(save ---- ,img_path)defmain():## 创建collectioncreate_collection()## 向量持久化to_milvus()if__name____main__:main()运行4.3 文搜图测试通过文本进行图像召回检索importbase64importrequestsfrompymilvusimportMilvusClientimportmatplotlib.pyplotaspltfromPILimportImage plt.rcParams[font.sans-serif][SimHei]clientMilvusClient(http://127.0.0.1:19530)collection_nameqwen3_vl_testdefencode_image(image_path):withopen(image_path,rb)asimage_file:returnbase64.b64encode(image_file.read()).decode(utf-8)defembed(textNone,imageNone):ifnottextandnotimage:raiseException(embed content is empty)ifimage:imageencode_image(image)responserequests.post(http://127.0.0.1:8848/v1/embeddings,json{text:text,image:image})returnresponse.json()[data]defmain():whileTrue:questioninput(请输入)ifnotquestion:passifquestionq:breakvecembed(textquestion)resclient.search(collection_name,data[vec],limit2,output_fields[content])plt.figure()plt.axis(off)plt.title(f输入问题{question},fontsize20,fontweightbold)forindex,iteminenumerate(res[0]):img_nameitem[entity][content]plt.subplot(1,2,index1)imageImage.open(fimages/{img_name})plt.imshow(image)plt.show()if__name____main__:main()运行后在控制台输入问题
阅读完成 · 觉得有帮助？
给了先后顺序)
