Fluents.ai and HighLevel - Workflow Automation

Dec 9, 2024

Introduction

In the modern landscape of customer engagement, businesses face increasing demands to respond promptly to leads and provide personalized interactions. Fluents.ai, an AI-powered voice agent platform, addresses these needs by automating and enhancing customer interactions. By leveraging advanced voice technology, Fluents.ai ensures swift responses to inquiries, enabling businesses to engage leads effectively and at scale.

HighLevel, a comprehensive CRM and marketing automation tool, empowers businesses to streamline workflow creation, optimize lead management, and improve customer retention. With its robust automation capabilities and user-friendly interface, HighLevel enables organizations to create tailored workflows that connect with customers at every stage of the sales funnel.

This document highlights the integration of Fluents.ai with HighLevel to demonstrate a seamless process for enhancing lead conversion. By leveraging HighLevel’s workflow automation and Fluents.ai’s AI-driven voice capabilities, businesses can create a powerful system to engage leads immediately, gather valuable insights, and take actionable steps based on those insights.

The showcase involves two workflows:

  1. Workflow 1: A lead fills out a form in HighLevel, triggering a webhook to Fluents.ai. The webhook initiates an immediate call from an AI voice agent, which interacts with the lead to gather information.

  2. Workflow 2: Once the call concludes, Fluents.ai sends a callback to HighLevel, which updates the CRM with a note containing the raw transcript. In a more advanced implementation, the transcript data could be parsed to update profiles, assign leads to human agents, schedule appointments, or trigger further actions.

Setup Fluents.ai

To get started with Fluents.ai, the first step is to sign up on the platform if you haven't already. You can create an account or log in using the following link:
Fluents.ai Login

For a successful integration, you will need the following information:

  • Agent ID: This identifies the agent that will respond to calls.

  • API Key: Required to authenticate requests to the Fluents.ai API.

  • Phone Number: The phone number associated with the agent for initiating calls.

If you are already familiar with Fluents.ai and have these details ready, you can proceed to the next section.

Fluents.ai main page:

Agent Creation

The first step in the setup process is to create an agent. An agent is an entity in Fluents.ai that requires the following components:

  • Voice: The voice provider that will be used for interactions. Fluents.ai currently supports four voice providers:

    • Azure

    • Eleven Labs

    • Play HT

    • Rime

More voice providers will be available in the future, and there will also be an option to import your own voice directly from a provider.

  • Prompt: The prompt defines the responses and behavior of the agent during calls or interactions.

Create a Voice

For this integration, we chose to use Eleven Labs - Charlie Voice, a soft voice with a hint of a British accent. You are welcome to experiment with other voices as well to find the best fit for your use case.

Follow these steps to create a voice:

Create a Prompt

The prompt is the heart and soul of the agent, defining how the agent will sound and behave during the interaction. It can also include metadata from the context by using bracketed variables, such as {variable_name}.

For this integration, we will gather the first name of the lead and their issue (if provided) within the prompt to give the agent context for the conversation.

For this integration, we used the following prompt:

You are an agent calling a lead after they submitted a form on our website. Your goal is to gather more information about their use case. The form may contain additional information, which will be enclosed in <> and represented as {variable_name}. Make sure to ask the following questions:

  • Confirm their name: {fname}

  • Confirm the description of the issue: {description} (rephrase if necessary)

  • Ask how they found us

  • Ask what interests them in the proposed services

Both {fname} and {description} will be dynamically populated at runtime based on the data submitted via the form in Highlevel.

Follow those steps to create a prompt:

Create the Agent

Once you have configured the voice and the prompt, it's time to create the agent.

Here the steps:

Buy or Import a Phone Number

The final step in setting up your agent is linking it to a phone number. Fluents.ai allows you to either purchase a new phone number or import an existing one to be used with your agent.

Here the steps to buy a phone number:

Gather the Required Information

Before proceeding with the integration, you will need the following essential details:

  1. Agent ID:
    This is the unique identifier for the agent you’ve created on Fluents.ai. The agent ID will be used to reference the specific agent you want to interact with during the workflow.

  2. Phone Number:
    The phone number linked to the agent will be required for making outbound calls. This can either be a newly purchased number or an imported number, as discussed earlier.

  3. API Key:
    To authenticate and interact with the Fluents.ai platform programmatically, you will need your API key. This key is essential for secure communication between Highlevel and Fluents.ai.

Once you’ve gathered this information, you are ready to proceed with setting up the integration between Fluents.ai and Highlevel.

Agent ID:

Phone number:

API Key:

That concludes the Fluents.ai overview, along with the necessary setup and requirements for integration.


Setup HighLevel

If you haven’t already, sign up for HighLevel here.

Create a Sub Account

Start by creating a sub-account in HighLevel. Sub accounts are essential for organizing workflows and managing specific client data.

Here the steps to create a basic sub account:

Then:

And finally go to your new sub account:

Create a Form

Next, create a form in HighLevel. This form will serve as the trigger point for the workflow that integrates with Fluents.ai.

Here are the steps:

A new page to create our form just popped up.
First, let’s remove the consent form and replace it with a multiline input box where the lead can describe their issue.

Feel free to customize the form as needed. For this integration, we require the following fields:

  • First Name

  • Last Name

  • Phone Number

  • Email Address

  • Issue Description (multiline input box)

These fields will ensure we have all the necessary information for a smooth integration with Fluents.ai.

Extra step, we decided to create and host the form using the embedded code provided by Highlevel.

We now have everything needed on HighLevel. The next step involves setting up the middleware to connect Fluents.ai with HighLevel.

Highlevel Workflows

This section outlines the process of creating workflows and integrating them with Fluents.ai. Key concepts will be introduced as needed.

1st Workflow - Form Submission

The first workflow is triggered when a lead fills out and submits a designated form. This workflow has two simple steps:

  1. Form Submission: The lead completes the form and clicks the submit button.

  2. Middleware API Call: The workflow sends the form data to a middleware backend API. This call initializes a phone call using the provided information.

Here the steps to create the 1st workflow:

2nd Workflow - Callback Integration from Fluents.ai to Highlevel

The second workflow is triggered by an inbound webhook after a call has been processed by Fluents.ai. This workflow involves three steps:

  1. Inbound Webhook Detection: The middleware backend API sends a payload to the Highlevel webhook once the call is completed.

  2. Contact Lookup: The workflow uses the contact_id from the payload to find the corresponding contact.

  3. Note Creation: A note is added to the contact's details, containing the raw transcript of the call.


Middleware

The next step is to develop the middleware that bridges Fluents.ai and HighLevel. This middleware will ensure that when a form is submitted and a workflow is triggered in HighLevel, a call is initiated via Fluents.ai.

The middleware is straightforward:

  • It consists of a basic controller (for demonstration purposes) with two endpoints:

    1. Submit Form – Handles incoming webhook requests from HighLevel when the form is submitted.

    2. Callback – Receives responses from Fluents.ai once the call is completed.

  • A service is responsible for processing the call, managing data exchange between the two platforms.

Update the Agent

Overview:
This section outlines the steps required to update the agent entity with webhook settings. While this process will soon be obsolete with the upcoming Fluents.ai platform features, it remains essential for the current integration.

Agent Webhook Overview

The agent entity in Fluents.ai supports a variety of configurable parameters, including webhooks.
A webhook configuration typically requires the following:

  • URL: The endpoint to which events will be sent.

  • Method: The HTTP method to use (default is POST).

  • Subscriptions: A list of events the user wishes to subscribe to.

Fluents.ai supports several event types, such as:

  • event_phone_call_did_not_connect

  • event_human_detection

  • event_phone_call_ended

For this integration, we are particularly interested in the event_phone_call_ended event. More details about webhook configuration and event types can be found here.

Webhook Configuration (Current Requirements)

Since webhook configuration is not yet available directly via the Fluents.ai platform, this must be done programmatically. Below are the required parameters for our current setup:

Once the platform supports webhook setup natively, this step will no longer require manual intervention.

Here the code snippet:

// Fetches details of a specific agent from Fluents.ai
private async getAgent(
  apiKey: string, // API key for authorization
  apiUrl: string, // Base URL of the Fluents.ai API
  agentId: string, // ID of the agent to retrieve
): Promise<AgentDto> {
  const bearer = `Bearer ${apiKey}`; // Construct the Authorization header
  const endpoint = `${apiUrl}/agents?id=${agentId}`; // Build the API endpoint
  const options = {
	method: 'GET', // HTTP method for fetching data
	headers: {
  	Authorization: bearer, // Include API key for authentication
  	'Content-Type': 'application/json', // Specify JSON content type
	},
  };

  try {
	// Make the API request to fetch agent data
	const res = await fetch(endpoint, options);

	// Check if the response status is not OK (e.g., 4xx, 5xx errors)
	if (!res.ok) {
  	const errorText = await res.text(); // Read the response body as text for error details
  	throw new Error(`Error fetching agent data: ${errorText}`); // Throw detailed error
	}

	// Parse the response JSON into a plain object
	const responseBody = await res.json();

	// Convert the response into an AgentDto instance
	const agentDto = plainToInstance(AgentDto, responseBody);

	// Validate the AgentDto instance against its schema or rules
	const errors = await validate(agentDto);
	if (errors.length > 0) {
  	console.error('Validation errors:', errors); // Log validation errors for debugging
  	throw new Error('Validation failed for AgentDto'); // Stop execution on validation failure
	}

	return agentDto; // Return the validated AgentDto instance
  } catch (error) {
	console.error('Error in getAgent:', error); // Log errors during the API call or processing
	throw error; // Rethrow the error for higher-level handling
  }
}

// Updates the agent's webhook configuration
async updateAgent() {
  const apiKey = process.env.API_KEY; // Retrieve API key from environment variables
  const apiUrl = process.env.API_URL; // Retrieve base URL from environment variables
  const agentId = process.env.AGENT_ID; // Retrieve agent ID from environment variables

  // Fetch the current agent data
  const agentDto = await this.getAgent(apiKey, apiUrl, agentId);

  // Create a new webhook DTO with the desired configuration
  const webhookDto = new WebhookDto();
  webhookDto.subscriptions = [SubscriptionEvent.EVENT_PHONE_CALL_ENDED]; // Subscribe to specific events
  webhookDto.url = process.env.CALLBACK_URL; // Set callback URL from environment variables
  agentDto.webhook = webhookDto; // Assign the webhook configuration to the agent

  const bearer = `Bearer ${apiKey}`; // Construct the Authorization header
  const endpoint = `${apiUrl}/agents/update?id=${agentId}`; // Build the API endpoint for updating agents
  const options = {
	method: 'POST', // HTTP method for sending data
	headers: {
  	Authorization: bearer, // Include API key for authentication
  	'Content-Type': 'application/json', // Specify JSON content type
	},
	body: JSON.stringify(agentDto), // Serialize the updated agent DTO to JSON
  };

  try {
	// Call the API to update the agent configuration, with retry logic for robustness
	await this.fetchWithRetries(endpoint, options);
  } catch (e) {
	console.log('Error occurred while updating agent:', e.message); // Log error message
	console.log('Stack trace:', e.stack); // Log stack trace for debugging
	throw e; // Rethrow the error for further handling
  }
}


Once the agent's webhook configuration is successfully updated:

  1. Webhook Configuration: The agent now has a webhook registered to the specified CALLBACK_URL with the event subscription for EVENT_PHONE_CALL_ENDED.

  2. Triggering Webhook:

    • Any subsequent calls made using this agent will automatically trigger a webhook event when the call ends.

    • Fluents.ai will send a POST request to the configured CALLBACK_URL with details of the event_phone_call_ended.

  3. Use Case: This enables seamless integration between Fluents.ai and your middleware, allowing your application to:

    • Log call completion events.

    • Update lead statuses.

    • Perform follow-up actions, such as sending emails or updating your CRM.

  4. Future-proofing: While webhook configuration might soon be available directly via the Fluents.ai UI, this programmatic approach ensures current integrations are functional and scalable.

Workflow diagram

Here is a flow diagram representing the chain of events after a form is submitted:

Here is a breakdown of each part of the diagram:

Form Completed and Submitted

This is the entry point of the workflow. A lead fills out and submits the form, providing information with the intent to connect with your organization.

Calling an External Webhook to the Middleware (submitForm)

HighLevel triggers a call to our middleware by sending a webhook request. This request carries the lead's details, which include:

  • first_name: The lead's first name.

  • last_name: The lead's last name.

  • email: The lead's email address.

  • phone: The lead's phone number.

  • issue: A brief description of the lead's problem (this field is optional).

Process the Payload to Create a Call

Using the payload provided by HighLevel and the necessary credentials and information to interact with the Fluents.ai core API server (such as the agent ID, API key, and phone number), the middleware prepares a new payload to trigger the call.

This step includes:

  • Creating Context for the Agent:

    • The context contains dynamic data like the lead's first name (first_name) and a brief description (description).

    • For this integration, the context also includes the contact_id for later use.

⚠️ Note: While storing contact_id in the agent's context works for this demonstration, it is not the recommended approach. Best practices suggest storing such information in a dedicated database for better scalability, security, and maintainability.

Here the cloud function used:

import fetch from 'node-fetch';

/**
* Process a lead by making an API call to create a call with the provided contact information.
*
* @param {object} req - The HTTP request object containing the lead data in the body.
* @param {object} res - The HTTP response object used to send back the result.
*/
export const processLead = async (req, res) => {
 const contact = req.body;
 console.log('Received request body:', req.body);

 // Validate input data
 if (!contact?.phone || !contact?.first_name || !contact?.contact_id) {
   res.status(400).send({
     error: 'Invalid input data. Required: phone, first_name, contact_id.',
   });
   return;
 }

 // Load required configuration from environment variables
 const {
   API_KEY: apiKey,
   API_URL: apiUrl,
   AGENT_ID: agentId,
   PHONE_FROM: phone,
 } = process.env;

 // Validate required environment variables
 if (!apiKey || !apiUrl || !agentId || !phone) {
   console.error('Missing required environment variables.');
   res
     .status(500)
     .send({ error: 'Server configuration error. Please contact support.' });
   return;
 }

 // Prepare the payload for the API call
 const callDto = {
   from_number: phone,
   to_number: contact.phone,
   agent: agentId,
   context: {
     fname: contact.first_name,
     description: contact.Issue || '', // Optional field
     contactId: contact.contact_id,
   },
 };

 // API request configuration
 const endpoint = `${apiUrl}/calls/create`;
 const options = {
   method: 'POST',
   headers: {
     Authorization: `Bearer ${apiKey}`,
     'Content-Type': 'application/json',
   },
   body: JSON.stringify(callDto),
 };

 try {
   // Make the API call
   const response = await fetch(endpoint, options);

   if (!response.ok) {
     // Log detailed error response for debugging
     const errorText = await response.text();
     console.error('Failed to process lead:', response.status, errorText);
     res.status(response.status).send({ error: `API error: ${errorText}` });
     return;
   }

   // Success response
   res.status(200).send({ message: 'Lead processed successfully' });
 } catch (error) {
   // Handle unexpected errors
   console.error('Error occurred while processing lead:', error.message);
   console.error('Stack trace:', error.stack);
   res.status(500).send({ error: 'Internal Server Error' });
 }
};


For more information about the call endpoint. Here

API url: https://api.fluents.ai/v1

Fluents.ai Agent Calls the Lead

  1. Initiating the Call:

    • Once the Fluents.ai core API server receives the API call, it initiates the process to place a call to the lead's provided phone number.

  2. Lead Interaction:

    • The lead hears their phone ring.

    • If the lead answers, a conversation is initiated between the agent and the lead.

    • The agent:

      • Follow the predefined prompts.

      • Reacts dynamically to the lead’s inputs.

      • Collects as much relevant information as possible during the conversation.

  3. Call Completion:

    • The conversation ends when the lead hangs up or the agent determines the interaction is complete.

Middleware Callback is Triggered

Webhook Overview:

  • Event Trigger: Once the call ends, Fluents.ai core API server triggers a webhook for the event_phone_call_ended event.

  • Webhook POST Request: This webhook sends a POST request to the pre-configured callback URL.

Webhook Payload Structure:

  • call_id: The unique identifier of the call.

  • type: The type of event, in this case, event_phone_call_ended.

  • payload: Additional details related to the event, if applicable.

Process the Callback and Trigger Highlevel Inbound Webhook

After receiving the event_phone_call_ended webhook, the following actions are performed:

  1. Retrieve Call Details: We call the Fluents.ai core API server to fetch the call details using the provided call_id from the webhook.

  2. Extract Transcript: We extract the transcript of the conversation between the agent and the lead.

  3. Retrieve Contact Information: The contact_id, which was stored during the interaction, is retrieved.

  4. Create Payload: A payload is created containing the contact_id and the transcript of the conversation.

  5. Send to Highlevel: A POST request is sent to Highlevel’s inbound webhook to trigger the second workflow. This workflow uses the contact_id to locate the corresponding contact and create a note with the conversation transcript.

Here the cloud function used:

import fetch from 'node-fetch';

/**
* Handles the callback by fetching call data and sending it to the  webhook.
*
* @param {object} req - The HTTP request object containing callback data in the body.
* @param {object} res - The HTTP response object used to send the result.
*/
export const callback = async (req, res) => {
 try {
   const callbackDto = req.body;

   // Validate input payload
   if (!callbackDto?.call_id) {
     return res
       .status(400)
       .send({ error: 'Invalid payload. "call_id" is required.' });
   }

   // Load environment variables
   const {
     API_KEY: apiKey,
     API_URL: apiUrl,
     URL_CALLBACK: urlWebhook,
   } = process.env;

   // Validate required environment variables
   if (!apiKey || !apiUrl || !urlWebhook) {
     console.error('Missing required configuration in environment variables.');
     return res
       .status(500)
       .send({ error: 'Server configuration error. Please contact support.' });
   }

   const callId = callbackDto.call_id;

   // Fetch call data
   const callDataEndpoint = `${apiUrl}/calls?id=${callId}`;
   const callDataOptions = {
     method: 'GET',
     headers: {
       Authorization: `Bearer ${apiKey}`,
       'Content-Type': 'application/json',
     },
   };

   const callResponse = await fetch(callDataEndpoint, callDataOptions);

   if (!callResponse.ok) {
     const errorDetail = await callResponse.text();
     console.error(
       `Error fetching call data: Status ${callResponse.status} - ${errorDetail}`
     );
     return res.status(500).send({ error: 'Failed to fetch call data' });
   }

   const callData = await callResponse.json();

   // Prepare payload
   const dto = {
     contact_id: callData.context?.contactId || '',
     raw_transcript: callData.transcript || 'No transcript found',
   };

   // Send data to the webhook
   const Options = {
     method: 'POST',
     headers: {
       Authorization: `Bearer ${apiKey}`,
       'Content-Type': 'application/json',
     },
     body: JSON.stringify(dto),
   };

   console.log('Sending data to the webhook:', dto);
   const Response = await fetch(urlWebhook, Options);

   if (!Response.ok) {
     const errorDetail = await Response.text();
     console.error(
       `Error sending data to the webhook: Status ${Response.status} - ${errorDetail}`
     );
     return res
       .status(500)
       .send({ error: 'Failed to send data to the webhook' });
   }

   console.log('Response Status:', Response.status);
   res.status(200).send({ message: 'Callback processed successfully' });
 } catch (error) {
   console.error('Unexpected error while processing callback:', error.message);
   console.error('Stack trace:', error.stack);
   res.status(500).send({ error: 'Internal Server Error' });
 }
};



The Result

Now that the loop is closed, let’s verify if the contact has a note or more containing the transcript.

This step involves checking the contact's profile within the HighLevel CRM to ensure that:

  • The transcript of the conversation is properly recorded as a note.

  • All relevant details from the workflow are accurately reflected.

By doing so, we confirm that the process is working end-to-end and the data flow is seamless

As we can see, there are multiple notes recorded. This indicates that the lead submitted the form more than once, and each submission triggered a separate workflow.

This confirms that:

  • The workflows handle repeated submissions correctly.

  • Each transcript is logged as a separate note, ensuring comprehensive tracking of all interactions.

Improvements

This solution serves as a foundational integration, demonstrating that Fluents.ai and a tool like HighLevel can collaborate effectively. However, there are opportunities for enhancement to make the workflows more robust and valuable.

Suggested Improvements:

1. Enhancing the 1st Workflow:

  • Retrieve the contact's details upon form submission.

  • Check for any existing notes associated with the contact.

  • Send these notes to the middleware and include them in the context.

  • This would empower the agent with all relevant information, enabling them to continue the conversation seamlessly and provide better assistance.

2. Post-Processing in the Middleware:

  • When a callback from Fluents.ai is triggered, process the transcript further.

  • Extract key information from the raw transcript, such as intent, critical details, or specific action items.

  • Format this information into a more structured and detailed note, rather than relying on raw transcripts.

These enhancements would:

  • Improve the agent's context during conversations, leading to better interactions with leads.

  • Generate richer, more actionable notes for the HighLevel workflows, adding value to the contact's history and ensuring comprehensive documentation.

Fluents | Copyright© 2024

Free Trial