Configuring a Data Science Connector in Search with OpenSearch

• To use a Data Science connector with OCI Search with OpenSearch, you need a cluster configured to use OpenSearch version 2.11 or newer. By default, new clusters are configured to use version 2.11. To create a cluster, see Creating a Search with OpenSearch Cluster.

  For existing clusters configured for version 2.3, you can perform an inline upgrade to version 2.11. For more information, see Upgrading Search with OpenSearch Cluster Software.

  To upgrade existing clusters configured for version 1.2.3 to 2.11, you need to use the upgrade process described in Upgrading Search with OpenSearch Cluster Software.

• If the OpenSearch cluster is in a different tenancy than the Data Science endpoint tenancy, you need to create policies in both tenancies to grant access to Data Science resources.

  The following policy examples includes the required permissions. To use these examples, replace <caller_tenancy_name> with the name of the tenancy for the OpenSearch cluster, and replace <resource_host_tenancy_name> with the name of the tenancy for the Data Science endpoint.

  • Policy for Data Science tenancy:

    define tenancy <caller_tenancy_name> as <caller_tenancy_ocid>
    admit any-user of tenancy <caller_tenancy_name> to {DATA_SCIENCE_MODEL_DEPLOYMENT_PREDICT} in tenancy

  • Policy for OpenSearch cluster tenancy:

    define tenancy <resource_host_tenancy_name> as <resource_host_tenancy_ocid>
    endorse any-user to {DATA_SCIENCE_MODEL_DEPLOYMENT_PREDICT} in tenancy <resource_host_tenancy_name>

  If you're new to policies, see Getting Started with Policies and Common Policies.

Use the settings operation of the Cluster APIs to configure the recommended cluster settings, as follows:

PUT _cluster/settings
{
  "persistent": {
    "plugins": {
      "ml_commons": {
        "only_run_on_ml_node": "false",
        "model_access_control_enabled": "true",
        "native_memory_threshold": "99",
        "rag_pipeline_feature_enabled": "true",
        "memory_feature_enabled": "true",
        "allow_registering_model_via_local_file": "true",
        "allow_registering_model_via_url": "true",
        "model_auto_redeploy.enable":"true",
        "model_auto_redeploy.lifetime_retry_times": 10
      }
    }
  }
}

Create the Data Science connector using the following payload:

POST _plugins/_ml/connectors/_create
{
   "name": "DS Chat Connector",
   "description": "Check errors in logs",
   "version": 2,
   "protocol": "oci_sigv1",
   "parameters": {
       "endpoint": "<model_deployment_endpoint>",
        "modelId": "<model_deployment_OCID>",
       "auth_type": "resource_principal"
   },
   "credential": {
   },
   "actions": [
       {
           "action_type": "predict",
           "method": "POST",
           "url": "https://${parameters.endpoint}/${parameters.modelId}/predict",
           "request_body": "{\"model\": \"/opt/ds/model/deployed_model\", \"prompt\" : \"${parameters.prompt}\", \"max_tokens\":250,  \"temperature\": 0.7, \"top_p\":0.8, \"stream\" : false}",
           "post_process_function": "def text = params['choices'][0]['text'].replace('\n', '\\\\n');\n return '{\"name\":\"response\",\"dataAsMap\":{\"inferenceResponse\":{\"generatedTexts\":[{\"text\":\"' + text + '\"}]}}}'"
       }
   ]
}

Replace <model_deployment_endpoint> and <model_deployment_OCID> with the actual values for your Data Science model deployment, for example:

...
"endpoint": "modeldeployment.us-ashburn-1.oci.customer-oci.com"
"modelId": "ocid1.datasciencemodeldeployment.oc1.IAD.examplesmtpsuqmoy4m5cvblu..."
...

Make note of the <connector_ID> returned in the response:

{
  "connector_id": "<connector_ID>",
}

Register a model group for the connector using the register operation in the Model Group APIs, as shown in the following example:

POST /_plugins/_ml/model_groups/_register
{
   "name": "<model_group_name>",
   "description": "<model_group_description>"
}

Make note of the <model_group_ID returned in the response:

{
  "model_group_id": "<model_group_ID>",
  "status": "CREATED"
}

Register the model using the <model_group_ID> and <connector_ID> from the preview step, as shown in the following example:

POST /_plugins/_ml/models/_register
{
   "name": "oci-genai-test",
   "function_name": "remote",
   "model_group_id": "<model_group_ID>",
   "description": "test model",
   "connector_id": "<connector_ID>"
 }

Deploy the model, as shown in the following example:

POST /_plugins/_ml/models/<model_ID>/_deploy

Create a RAG pipeline using the <model_ID> from the previous step, as shown in the following example:

PUT /_search/pipeline/<pipeline_name>
{
  "response_processors": [
    {
      "retrieval_augmented_generation": {
        "tag": "genai_pipeline_demo",
        "description": "Demo pipeline Using Genai Connector",
        "model_id": "<model_ID>",
        "context_field_list": ["<text_field_name>"],
        "system_prompt": "You are a helpful assistant",
        "user_instructions": "Generate a concise and informative answer for the given question"
      }
    }
  ]
}

You can specify one or more text field names for "context_field_list", separate the values with a comma, for example:

"context_field_list" : ["value1", "value2",…,"valueN"],

Values specified for "context_field_list" must match a field or fields of interest in the target document index.