ChatGPT recently became available as an app in the App Store. This is only available in the US. I am working on improving my prompt engineering skills, and an app would be ideal to quickly try something on the go. I also found out that the Azure OpenAI service became available as private preview in regions West Europe and France Central. So we will be creating our own ChatGPT app, built on Azure OpenAI service.
Challenge Objectives
🎯 Deploy Azure OpenAI Service
🎯 Build a Custom Connector for the Azure service
🎯 Use Custom Connector in a Canvas App
🎯 Make a responsive ChatGPT Canvas App
Introduction
In the previous challenge, we've connected Power Virtual Agents to the OpenAI API. I chose for the OpenAI API, as this was what I had access to at that time. Ideally, we want these The AI models are the same, so for learning purposes, that's all fine. I also mentioned that within enterprise scenarios, using the model hosted on Azure is by far the recommended option.
Because it is a hot topic and the disruptive potential, I think it's valuable to all Power Apps developers to know how to include these capabilities, as we can expect lots of requests related to ChatGPT.
I generally prefer not to use private preview functionality, as it requires you to ask permission to these resources, which adds waiting time to your process. I also try not to use any paid services from Azure, as this will result in some dropping off due to the involved costs.
The Azure OpenAI Service requires both submitting a request form and Azure costs. The reason for this is that the access to this service will grow rapidly, and the costs of the service we are using will be literally a few cents.
It is good to be aware of these prerequisites, before proceeding.
Deploy Azure OpenAI Service
At the time of writing, Azure OpenAI Service is in private preview. It might be open to all when you are reading this, so let's check that first.
go to portal.azure.com
Navigate to Subscriptions
Check if you have an active subscription. If not, start one. Pick the Free trial if this is an option for you and you only want to use this to see the functionality. In all other cases, just select Pay-as-you-Go
Navigate to Resource Groups
Create a new Resource Group. Select your subcription, and give it a name (e.g. PowerPlatfromChallenge019)
Before you select a region, check in which regions the Azure OpenAI Service is actually available
Select that region
Press Review + create
If the validation passed, create the resource group
If you are not familiar with subscriptions, resource groups, and resources, the image below might be helpful.
Resources belong to a resource group, which is similar to a solution. The subscription is there to manage the costs of one or more resource groups.
Now that we have a subscription and a resource group, it's time to see if the Azure OpenAI Service if available for you.
Open the recently created resource group
Select Create in the command bar
Search for Azure OpenAI
Select Create
If you will see the red info panel, it is still in private preview and you will need to request access. You can do that by clicking the link and filling in the request form. You will be notified when granted (or declined...).
If this is not there, you can just create it and you are all settled. If you did not find the resource, you've probably selected an unsupported region.
The deployment of this resource might take some time, unfortunately. For me, it took a few hours. Once you have the resource listed in your resource group, you will need to deploy an actual AI model. The Azure OpenAI service contains multiple models that can be used. The one we will be using is the gpt-35-turbo model, which is the model that you know from ChatGPT.
Click on your recently created Azrue OpenAI Service
Select the Deploy option
Press the Create new deployment button
Select the gpt-35-turbo model from the dropdown
Give it a name (e.g. GPT35Turbo)
From now on, you can start using the power of ChatGPT within your own Azure tenant. You can try it in playground. Note how the Playground has some pre-populated options for the system message. This is a basic form of Prompt Engineering.
Azure OpenAI Custom Connector
It is cool that we have the model running in our own Azure tenant. But we all know that connecting to a service with Power Apps requires connectors. We could dive into the documentation to make an HTTP call, or use it to create a custom connector. But there is some really good news!
You might be aware of the GitHub repo where all the Independent Publishers create a Pull Request to add their connector to the platform. The cool thing is that this repo also contains some pre-built custom connectors. The even cooler thing is that it contains a custom connector definition for Azure OpenAI service. The king of the coolness-hill is that it is extremely easy to add it to your solution, in order to start using this Service with Power Platform. Let's do that right away.
Go to make.powerautomate.com
Navigate to Data > Custom connectors
on the top right, select New custom connector
Select Import from GitHub
Select Custom for the Connector Type, master for Branch, and AzureOpenAIService for Connector
Press Create connector
That's the fastest I have ever created a custom connector. Unfortunately, I found out it was just there after creating a custom connector myself... But this connector is much richer in functionality, and stupid simple to create. Much better.
To use it, we should add a connection for the connector. It requires an API key, and an instance name.
You can find both on the Keys and Endpoints page, or on the essentials section on the Overview page. For the API key, you can copy either key 1 or key 2. The instance name is basically the name of your Azure OpenAI Service. the naming is part of your endpoint. you only have to specify your unique part of the endpoint, (in my case openaipowerbouwer). the documentation of the custom connector explains it in more detail, if required.
There is one more thing to know, before you are going to use the connector. Go to the custom connector, go to edit mode, and go to the test section.
You will see that most actions require a deployment-id. Here you should fill in the name of the AI model that we've deployed earlier. The good thing is that the custom connector does make use of dynamic dropdown type, which means that the available models will be shown in a nice fashion when using Power Automate. We don't have that functionality yet in Power Apps, so it is good to be aware of what this required parameter actually expects as an input.
Give it a spin in the test section. Hopefully you will get a 200 response. You can also try to create a flow with this connector, to see the dynamic dropdown type in action. Nice work by Daniel Laskewitz, Andrew Coates, and Robin Rosengrün. Thanks for saving us all valuable time!
Create an App
We will create a Power App, which will basically be our interface for the API. I will guide you to through the steps of using the Creator Kit and making it responsive.
Everything apart from Azure OpenAI has been covered before. So if you need more guidance, please browse through earlier challenges. Especially Challenge 007 and Challenge 008 are recommended, as they focus on the Creator Kit and Responsive Design.
Preparation
Make sure the Creator Kit is installed to your environment
Create a solution. I named mine ChatGPT. You could also place the earlier created custom connector in it
Add the existing canvas app Canvas Template to your solution
Go to edit mode, and directly press save as. Name it ChatGPT
Once saved, go to Settings, and turn make sure that Scale to fit, Lock aspect ratio, and Lock orientation are turned off. This will allow for a responsive app that can also rotate. Especially convenient on tablets. For phones not as much, but phone operating systems can lock the orientation too.
Remove the Canvas Template app from the solution
Add Tables
If you have been using ChatGPT before, you might have notices that earlier chats are stored for later use. We want the same functionality, so we will need some tables to store it. The first one is the Chat table. The default name column is sufficient for this one. The second table is the Message table, which will contain all the messages from the chat. There are two options of completion actions in the custom connector. I will use the Create a completion action, as I had some issues with the Chat Completion (Preview) option. Every message in the chat will start with a question and end with an answer. That's why I will create these columns in my Messages table.
If you use the chat playground view code option, you will see the other option. If you wish to use this option, you should store the role and content in corresponding columns. This will result in a separate record for the both the question and the answer. These options are also explained in the custom connector documentation. I will go for the question answer pair.
When creating the table, we will set the data type of the name column to Auto Number. , and add a Question and Answer column. For simplicity I made these both text fields. Make sure to set the Maxinum character count to its maximum potential (850).
Create another column (Chat) which will be a lookup to the Chats table. Set the relationship between the messages and the Chats as Parental. This will make sure that when you delete a chat, all corresponding messages will be deleted too.
Add the tables to your app.
Create the responsive containers
Create a new screen named Chat Screen
Select the App in the tree view, and set the StartScreen to 'Chat Screen'
Add a container to the screen and name it conLeft. X = 0, Y = 0, Height = App.Height. Set the Width to the snippet below.
Add a container to the screen and name it conRight. X = conLeft.Width, Y = 0, Height = App.Height, Width = App.Width - Self.X.
Switch(
Parent.Size,
ScreenSize.Small,
Parent.Width,
280
)Fill conLeft
The conLeft should look like the image above. The control's I've used can be found below.
ckico is the icon from the Creator Kit. The top one is of type Action Button. The OnChange is set to the snippet below to add a new chat.
Patch(Chats, Defaults(Chats), {Name: "New chat"})galChats will show all chats. The OnSelect of galChats is set to the snippet below.
UpdateContext({lclItemSelected: true})This is required to switch between the left and right container on a phone. To make this work, update the Width property of conLeft to:
Switch(
Parent.Size,
ScreenSize.Small,
If(lclItemSelected, 0, Parent.Width),
280
)The ckicoDeleteChat control is there to delete the chat. I don't like to delete it directly, so I've added a dialog to the screen (outside of the containers). The delete icon will set a local variable called deleteMode to true. The visibility of the dialog is set to this variable. The OnButtonSelect property of the dialog is set to the snippet below. I am confident this should be enough information.
Switch(
Self.SelectedButton.Label,
"Ok",
Remove(Chats, galChats.Selected);
UpdateContext({deleteMode: false}),
"Cancel",
UpdateContext({deleteMode: false}),
Notify("An unsupported button has been pressed.", NotificationType.Warning)
)Fill conRight
The right container should look like the image above. It looks a bit boring, but it has some items there (see tree view below).
The first thing is a command bar. For now it only contains one button to navigate back to conLeft on Mobile devices. I did this intentionally, as I want to extend this app in the next challenge. To help you a bit I will give the OnSelect snippet of the command bar. The rest is up to you.
Switch( Self.Selected.ItemKey,
"back", UpdateContext({lclItemSelected: false}),
Notify("An unsupported button has been pressed.", NotificationType.Warning)
)The next piece is a vertical container. I opted for this container, as I want the chat to be in the middle with wider screens. This is quite common, as otherwise the lines will be really hard to read. How to set it in the middle dynamically is on you. I will give you the Width property. An additional hint, set the Justify vertical property to bottom and the horizontal one to strech.
If(ScreenSize = ScreenSize.Small, Parent.Width, Min(Parent.Width, 800))Then you will see another gallery. This will contain all the messages of the selected chat. I use the Flexible height gallery, as some messages can be longer than others. The label should be easy. The image for the question is set to User().Image. The snippet for the answer image is the ChatGPT icon, which is an SVG:
With(
{
varIconPath: "M37.5324 16.8707C37.9808 15.5241 38.1363 14.0974 37.9886 12.6859C37.8409 11.2744 37.3934 9.91076 36.676 8.68622C35.6126 6.83404 33.9882 5.3676 32.0373 4.4985C30.0864 3.62941 27.9098 3.40259 25.8215 3.85078C24.8796 2.7893 23.7219 1.94125 22.4257 1.36341C21.1295 0.785575 19.7249 0.491269 18.3058 0.500197C16.1708 0.495044 14.0893 1.16803 12.3614 2.42214C10.6335 3.67624 9.34853 5.44666 8.6917 7.47815C7.30085 7.76286 5.98686 8.3414 4.8377 9.17505C3.68854 10.0087 2.73073 11.0782 2.02839 12.312C0.956464 14.1591 0.498905 16.2988 0.721698 18.4228C0.944492 20.5467 1.83612 22.5449 3.268 24.1293C2.81966 25.4759 2.66413 26.9026 2.81182 28.3141C2.95951 29.7256 3.40701 31.0892 4.12437 32.3138C5.18791 34.1659 6.8123 35.6322 8.76321 36.5013C10.7141 37.3704 12.8907 37.5973 14.9789 37.1492C15.9208 38.2107 17.0786 39.0587 18.3747 39.6366C19.6709 40.2144 21.0755 40.5087 22.4946 40.4998C24.6307 40.5054 26.7133 39.8321 28.4418 38.5772C30.1704 37.3223 31.4556 35.5506 32.1119 33.5179C33.5027 33.2332 34.8167 32.6547 35.9659 31.821C37.115 30.9874 38.0728 29.9178 38.7752 28.684C39.8458 26.8371 40.3023 24.6979 40.0789 22.5748C39.8556 20.4517 38.9639 18.4544 37.5324 16.8707ZM22.4978 37.8849C20.7443 37.8874 19.0459 37.2733 17.6994 36.1501C17.7601 36.117 17.8666 36.0586 17.936 36.0161L25.9004 31.4156C26.1003 31.3019 26.2663 31.137 26.3813 30.9378C26.4964 30.7386 26.5563 30.5124 26.5549 30.2825V19.0542L29.9213 20.998C29.9389 21.0068 29.9541 21.0198 29.9656 21.0359C29.977 21.052 29.9842 21.0707 29.9867 21.0902V30.3889C29.9842 32.375 29.1946 34.2791 27.7909 35.6841C26.3872 37.0892 24.4838 37.8806 22.4978 37.8849ZM6.39227 31.0064C5.51397 29.4888 5.19742 27.7107 5.49804 25.9832C5.55718 26.0187 5.66048 26.0818 5.73461 26.1244L13.699 30.7248C13.8975 30.8408 14.1233 30.902 14.3532 30.902C14.583 30.902 14.8088 30.8408 15.0073 30.7248L24.731 25.1103V28.9979C24.7321 29.0177 24.7283 29.0376 24.7199 29.0556C24.7115 29.0736 24.6988 29.0893 24.6829 29.1012L16.6317 33.7497C14.9096 34.7416 12.8643 35.0097 10.9447 34.4954C9.02506 33.9811 7.38785 32.7263 6.39227 31.0064ZM4.29707 13.6194C5.17156 12.0998 6.55279 10.9364 8.19885 10.3327C8.19885 10.4013 8.19491 10.5228 8.19491 10.6071V19.808C8.19351 20.0378 8.25334 20.2638 8.36823 20.4629C8.48312 20.6619 8.64893 20.8267 8.84863 20.9404L18.5723 26.5542L15.206 28.4979C15.1894 28.5089 15.1703 28.5155 15.1505 28.5173C15.1307 28.5191 15.1107 28.516 15.0924 28.5082L7.04046 23.8557C5.32135 22.8601 4.06716 21.2235 3.55289 19.3046C3.03862 17.3858 3.30624 15.3413 4.29707 13.6194ZM31.955 20.0556L22.2312 14.4411L25.5976 12.4981C25.6142 12.4872 25.6333 12.4805 25.6531 12.4787C25.6729 12.4769 25.6928 12.4801 25.7111 12.4879L33.7631 17.1364C34.9967 17.849 36.0017 18.8982 36.6606 20.1613C37.3194 21.4244 37.6047 22.849 37.4832 24.2684C37.3617 25.6878 36.8382 27.0432 35.9743 28.1759C35.1103 29.3086 33.9415 30.1717 32.6047 30.6641C32.6047 30.5947 32.6047 30.4733 32.6047 30.3889V21.188C32.6066 20.9586 32.5474 20.7328 32.4332 20.5338C32.319 20.3348 32.154 20.1698 31.955 20.0556ZM35.3055 15.0128C35.2464 14.9765 35.1431 14.9142 35.069 14.8717L27.1045 10.2712C26.906 10.1554 26.6803 10.0943 26.4504 10.0943C26.2206 10.0943 25.9948 10.1554 25.7963 10.2712L16.0726 15.8858V11.9982C16.0715 11.9783 16.0753 11.9585 16.0837 11.9405C16.0921 11.9225 16.1048 11.9068 16.1207 11.8949L24.1719 7.25025C25.4053 6.53903 26.8158 6.19376 28.2383 6.25482C29.6608 6.31589 31.0364 6.78077 32.2044 7.59508C33.3723 8.40939 34.2842 9.53945 34.8334 10.8531C35.3826 12.1667 35.5464 13.6095 35.3055 15.0128ZM14.2424 21.9419L10.8752 19.9981C10.8576 19.9893 10.8423 19.9763 10.8309 19.9602C10.8195 19.9441 10.8122 19.9254 10.8098 19.9058V10.6071C10.8107 9.18295 11.2173 7.78848 11.9819 6.58696C12.7466 5.38544 13.8377 4.42659 15.1275 3.82264C16.4173 3.21869 17.8524 2.99464 19.2649 3.1767C20.6775 3.35876 22.0089 3.93941 23.1034 4.85067C23.0427 4.88379 22.937 4.94215 22.8668 4.98473L14.9024 9.58517C14.7025 9.69878 14.5366 9.86356 14.4215 10.0626C14.3065 10.2616 14.2466 10.4877 14.2479 10.7175L14.2424 21.9419ZM16.071 17.9991L20.4018 15.4978L24.7325 17.9975V22.9985L20.4018 25.4983L16.071 22.9985V17.9991Z",
varIconColor: AppTheme.palette.themePrimary
},
With(
{
varImageData: Concatenate(
"data:image/svg+xml;utf8, ",
EncodeUrl(
Concatenate(
"<svg viewBox='0 0 41 41' xmlns='http://www.w3.org/2000/svg'><path d='",
varIconPath,
"' fill='#",
varIconColor,
"'/></svg>"
)
)
)
},
varImageData
)
)I also added the progress indicator from the Creator Kit. It is set as indeterminate so it will mimic that ChatGPT is working on it. The visibility is set to IsBlank(ThisItem.Answer), which will result in it only being visible when then answer is empty.
The bottom section should be easy. The function of the button will be handled later. First we need some flows.
Create the Get-Completion flow
I have a preference for things in Power Automate, as I think that is just easier to develop and debug.
The action below is the centerpiece of this whole challenge. It requires a deployment ID and some messages to respond on. The beauty of ChatGPT is that it is conversations. That's why you can add multiple historical messages which are taken into account for the completion.
Because I want it to be reusable, I've added an environment variable to the solution that will contain the Deployment ID. instead of retrieving it from the Dataverse table every time the flow runs, I set a global variable in the App.OnStart. That variable is fed into this flow to minimize the calls and make the flow as quick as possible. The whole flow looks like this:
Be aware that I use the Power Apps (V2) version, which works a bit better in my experience than the original one. The original one is the default one, so you will have to change the trigger. The input variable DeploymentID has been explained. The other input variables will be fed from the app as well. The Chat contains the ID of the chat in which a message will be created (parental relation). This is used in the second action to filter the messages based on a specific chat.
That Filter property on the List rows | Messages action is always a bit tricky for me. For these type of query strings I like to use FetchXML Builder in XRMToolBox. You can build your query with an interface and convert it to Power Automate query parameters. Just replace the hard coded GUID with the Dynamic content value and you are settled. If prefer typing, make sure to use the system names of your column names.
The Select | Messages data operation is to cut off the unnecessary columns and rename them to what AzureOpenAIService expects (question & answer). The original names are in my case ppchal_question and ppchal_answer.
The Azure OpenAI Service action Create a completion is now a walk in the park. The only thing you have to be aware of is that you will have to set the Deployment parameter to a manual input, which will be your DeploymentID from your input variable. Also set the API version to 2023-03-15-Preview.
The answer from the previous action is wat we will feed back to the app.
Create the Get-Summary flow
A nice feature of ChatGPT is that you can retrieve older chats. We facilitate that with the dataverse tables. What ChatGPT does to make it convenient to know which chat is which, is naming the chat based on your first prompt. We want this too.
As you can see, this flow is quite straightforward. Again some input variables. What I do want to address is the system instructions parameter. This is something that can help quite a lot in prompt engineering. This is actually a first step of this. We will get to that in more detail in next challenge. But what you can see is that we don't just want an answer, but we want to summarize the question. Very powerful stuff. That answer (thus the summary) will be returned to the app.
Hook Everything together
Add the flows to your app. To trigger everything we have to only adjust the Send button. The snippet below is what I've added.
UpdateContext({lclSendingMessage: true, lclMessage: txtMessageInput.Value, lclMessageCount: CountRows(galMessages.AllItems), lclScope: "You are a helpful assistant"});
Reset(txtMessageInput);
Patch(Messages, Defaults(Messages), {Question: lclMessage, Chat: galChats.Selected});
Set(gblCompletionResponse, 'Get-Completion'.Run(galChats.Selected.Chat, gblDeploymentID, lclMessage, lclScope).answer);
Patch(Messages, Last(galMessages.AllItems), {Answer: gblCompletionResponse});
If(lclMessageCount = 0, Set(gblSummaryResponse, 'Get-Summary'.Run(gblDeploymentID, lclMessage).summary));
Patch(Chats, galChats.Selected, {Name:gblSummaryResponse});
UpdateContext({lclSendingMessage: false})As you can see, I set some local variables. The variable lclSendingMessage is used to disable other controls while the flow is running. I assume you can do this yourself. I save the massage so I can reset the text input field directly. I also check if this is the first message in this chat. This is used later to determine of the summary flow should be triggered. I also set the scope here. For now it is static, but this is what will evolve over time, when we will play around with prompt engineering.
Then the first 'real' action is patching a new record to the message table. This is to directly show the message. Because the answer is left empty, the progress indicator will show up.
Then the Get-Completion flow is kicked. The required properties are provided. The nice thing of the V2 trigger is that it will show you what input is asked for. Is is stored as text, text_1, text_2 etc, but a little to the right you can see which property is actually meant. Not perfect, but better than v1.
Once the answer is received, we update the record with a Patch function. Then the output of the CountRows() function from earlier is used to determine if the Get-Summary flow is required. within the If() function we also add the Patch function to update the chat record if the flow has run.
The last step is updating the variable that disabled all controls in the first step.
That's it. You now have your own ChatGPT app. Enjoy.
Additional Information
The good
As mention in the previous challenge, the benefit of using Azure OpenAI Service over OpenAI itself is that the AI model will run in your own tenant. Much more secure, especially for those business scenarios. Another advantage, it's actually really cheap. A ChatGPT subscription will set you back 20 dollars a month. I have made 5 cents (!!!) of Azure costs during the development and testing of this challenge.
The not-so-ideal (or just bad)
The first is obvious, and that's the availability. I needed to apply to get access to this Azure resource. That's it for now, but that will improve over time, so doable.
The app is responsive, su you can use it on your phone. But in all honesty, the app OpenAI has created will be better in user experience than what I've created.
During the development I ran into quite some issues. The one time the action in the custom connector works like a charm, the other time it just won't do anything. I presume this has to do with it being in private preview. Maybe it's just a lack of Nvidia racks in Microsoft datacenters. At least I hope. Because that will improve over time.
Key Takeaways
👉🏻 Azure OpenAI is in private preview 😞
👉🏻 There are custom connectors ready for us to use
👉🏻 Keep the app. We will continue developing it next month...