MilVus Schema
Table Of Contents:
- What Is A MilVus Schema?
- What Is Field Schema?
- Create A Field Schema?
- Supported Data Types.
- What Is Collection Schema?
- Create A Collection Schema.
(1) What Is A MilVus Schema?
- Schema is used to define the properties of a collection and the fields within.
- Schema is the blueprint of a collection.
- Schema is like a structure of a field or column.
(2) What Is A MilVus Schema?
- A field schema is the logical definition of a field.
- It is the first thing you need to define before defining a collection schema and managing collections.
Note:
- Milvus supports only one primary key field in a collection.
(3) Field Schema Property.
(4) Create A Field Schema.
- To reduce the complexity in data inserts, Milvus allows you to specify a default value for each scalar field during field schema creation, excluding the primary key field.
- This indicates that if you leave a field empty when inserting data, the default value you specified for this field applies.
from pymilvus import FieldSchema
id_field = FieldSchema(name="id", dtype=DataType.INT64, is_primary=True, description="primary id")
age_field = FieldSchema(name="age", dtype=DataType.INT64, description="age")
embedding_field = FieldSchema(name="embedding", dtype=DataType.FLOAT_VECTOR, dim=128, description="vector")
position_field = FieldSchema(name="position", dtype=DataType.VARCHAR, max_length=256, is_partition_key=True)
(5) Create A Field Schema With Default Field Value.
from pymilvus import FieldSchema
fields = [
FieldSchema(name="id", dtype=DataType.INT64, is_primary=True),
# configure default value `25` for field `age`
FieldSchema(name="age", dtype=DataType.INT64, default_value=25, description="age"),
embedding_field = FieldSchema(name="embedding", dtype=DataType.FLOAT_VECTOR, dim=128, description="vector")
]
(6) Supported Data Types
DataTypedefines the kind of data a field contains.- Different fields support different data types.
- JSON as a composite data type is available. A JSON field comprises key-value pairs.
- Each key is a string, and a value can be a number, string, boolean value, array, or list. For details, refer to JSON: a new data type.
(7) What Is A Collection Schema?
- A collection schema is the logical definition of a collection.
- Usually you need to define the field schema before defining a collection schema and managing collections.
(8) Create A Collection Schema.
- Define the field schemas before defining a collection schema.
from pymilvus import FieldSchema, CollectionSchema
id_field = FieldSchema(name="id", dtype=DataType.INT64, is_primary=True, description="primary id")
age_field = FieldSchema(name="age", dtype=DataType.INT64, description="age")
embedding_field = FieldSchema(name="embedding", dtype=DataType.FLOAT_VECTOR, dim=128, description="vector")
position_field = FieldSchema(name="position", dtype=DataType.VARCHAR, max_length=256, is_partition_key=True)
schema = CollectionSchema(fields=[id_field, age_field, embedding_field], auto_id=False, enable_dynamic_field=True, description="desc of a collection")
(9) Create A Collection From A Data Frame
- You can also create a collection with
Collection.construct_from_dataframe, which automatically generates a collection schema from DataFrame and creates a collection.
import pandas as pd
df = pd.DataFrame({
"id": [i for i in range(nb)],
"age": [random.randint(20, 40) for i in range(nb)],
"embedding": [[random.random() for _ in range(dim)] for _ in range(nb)],
"position": "test_pos"
})
collection, ins_res = Collection.construct_from_dataframe(
'my_collection',
df,
primary_field='id',
auto_id=False
)