Reverse Proxy with Node Red

Reverse Proxy with Node Red

To use Node-Red out in the free world it needs a chaperone, that happens to be reverse proxy at the moment. If you are starting off from the beginning I would probably recommend following this guide. That’s assuming you have an Ubuntu server and are running Node-Red from there and you don’t mind running Nginx. What if you already have a server that may or may not be Ubuntu. Maybe you don’t like Nginx or maybe you already have Apache2 running. Or what if you are running Node-Red on a Raspberry Pi? Then that guide doesn’t work necessarily.

I searched all over again (briefly) on setting up Node-Red with SSL and or reverse proxy and it seems everyone has done it but no one is willing to share steps to do it? Is it that easy that it doesn’t need to be said? I wish I had found a write up to point the way, as it is simple but not necessarily easy. Especially for the average or new RPi tinkerer.

This is how I got it working. There may be other ways, and better ways but this is what I found worked for me. YMMV.

 

Prerequisites

  • A TLD domain pointed at your server.
  • A server connected to your network running Apache2 (could be a Pi!)
  • Your server open to the world (port forwarding required)
  • Legit non-self-signed SSL certs. Let’s Encrypt is free.
  • mod_proxy installed and enabled on the Apache server, here’s a good guide (only step #1 is needed)
  • Node-Red, either on the same server or another (like a RPi) doesn’t matter (same network)
  • Don’t forget to enable Node-Red admin password protection too.

 

I am going to assume that you already own the domain and have it setup and working so that you can go to https://yourdomain.com and it pops up. HTTPS, with a green padlock. If not, you need to buy the domain and point the DNS to your servers IP and create SSL certs, port forward yada yada. I host a server at home, makes it easy. For example, example and example. Oh and here are some Let’s Encrypt examples for Nginx and Let’s Encrypt on Apache.

You will need to edit your sites-available conf files and add the stuff for reverse proxy. You need more than just Node-Reds base URL to have a reverse proxy. I found it was best/easiest (shit maybe its required I don’t know, I’m just dangerous) to add a line for each webpage I was going to use specifically. So far thats Twilio voice, sms and Amazon Alexa.

 

Here is my Ubuntu Apache2 site-available conf for reverse proxy:

 

I’ve got a reverse proxy for each call I am going to make within Node-Red when using Twilio. These need to be accessible from the free world (to Twilio or Amazon etc). If you need more pages available to the world toss em in.

For example https://mydomain.com/twiliosms is what would be available to the world now.

Restart apache and you should be good to go.

 

Securing Node-Red
https://hoast.dk/wordpress/2015/10/24/securing-node-red/
https://www.hardill.me.uk/wordpress/2015/05/11/securing-node-red/
https://nodered.org/docs/security
https://nodered.org/docs/configuration

Node-Red-Admin (to hash a password)
https://nodered.org/docs/node-red-admin

Adding SSL to Node-Red
http://industrialinternet.co.uk/node-red/adding-https-ssl-to-node-red/

Let’s Encrypt SSL Certificates
https://daplie.com/articles/lets-encrypt-on-raspberry-pi/
https://www.linode.com/docs/security/ssl/install-lets-encrypt-to-create-ssl-certificates

https://gist.github.com/jhaury/10508bfef231bc09c4eae4723645b4f0

Node-Red IVR with Twilio

Node-Red IVR with Twilio

A while back I posted about using Twilio and Node-Red to send/receive SMS messages. I was fiddling around with some older project stuff (uncompleted) earlier today and bounced into the IVR setup I had. Wondering if it was working I called it, and no it wasn’t. So I had to fix it. It took me a minute to come across all the related pages needed to get it working again so I decided to compile them here.

What is an IVR?

Well if you Google IVR you get: Interactive voice response (IVR) is a technology that allows a computer to interact with humans through the use of voice and DTMF tones input via keypad.

And that basically sums it up. It allows you the person, to call a computer and have the computer know what you want when pressing 1, or 5. Similar to a phone switching system but they are usually used to report some kind of information instead of route calls. IVRs basically cut out someones job once upon a time.

A better example is of a field technician calling an IVR system to read back what signals were received on the clients security system he just tested.

What we are doing here?

Were are going to get a basic IVR running on our Node-Red server to interact with and read back data or to activate/deactivate things over MQTT. If you would rather (and/or) use SMS to achieve kinda the same thing read here.

Prerequisites

 

The Flow!

the flow

Here is the glorious flow!

You will need to create a global and have its value set so the IVR can read it back (in this example).

Drop a line in the comments if you have a question.

 

What helped:

https://flows.nodered.org/node/node-red-node-twilio
old outdated flow https://flows.nodered.org/flow/637b5f6128a8d423503f
https://groups.google.com/d/topic/node-red/yAEx5k2UHJw/discussion

 

SEND IT.

Using Amazon Polly with Node-Red

Using Amazon Polly with Node-Red

Amazon Polly and Node-Red. This was far easier to get going than any Alexa skill I tried to do. I just came across this the other day on Scargill’s Tech blog.

Amazon Polly is a service that turns text into lifelike speech. … With Amazon Polly, you only pay for the number of characters you convert to speech, and you can save and replay Amazon Polly’s generated speech.”

Basically you feed Polly SSML text and it returns it in spoken form as an MP3 (or other format). Sweet, I can give my server/Node-Red a voice! Not sure my wife will like this but it will actually server a valid purpose. I was admitted to the hospital (again) for a short three-day stay recently and while I was admitted (and loaded up on drugs) my server closet reached my maximum set temperature… and issued the text alert 187 times. Now I can have the server annunciate that the closet needs to be opened. I could add my wife’s number to the SMS list but that will fail, lol. Simply saving the MP3 on the server and having it be played back works just great.

I am sure you can tap into the API many other ways, and use Polly many other ways, but the simplest way was my path. I just logged into the Amazon Polly Developer console and slapped some text in the input box. It spit out an MP3 and I saved it. The catch here is that the free tier of the Polly program only lasts 12 months, then you get charged. Its super cheap though, $4 per 1 million characters. The free tier is 4 million per month (for 12 months). From what I have read Polly will search the cloud to see if the file already exists. If it does it pulls it down, if it doesn’t it creates it (and counts agains the character limit) and pulls it down. Is this for “the cloud” or your account? Dunno. Still pretty cheap.

I have circumvented the whole thing (for now). I don’t really plan on (or have the need to) generating Polly files on the fly. What works for me is as I said, just slapping a ton of phrases I think I’ll need into Polly and saving the files for later. I’m cool with that.

But, there is a node for NR that will generate Polly files for you on the fly, and/or pull down the ones already generated. I mentioned I found Polly on Scargill’s blog, he also used NR to play with Polly. He had to do it the hard way. Three days ago the node was made. Definitely check out his blog even if you just use the node. Its all good info.

Install node-red-contrib-polly-tts

https://flows.nodered.org/node/node-red-contrib-polly-tts
https://www.npmjs.com/package/node-red-contrib-polly-tts

npm install node-red-contrib-polly-tts

The node looks pretty simply to use (I have not used it). You need to add your Amazon Polly API credentials to the node. The input gets TTS’d and the file saved. If the text has already been converted no Polly call is made and just the file served. So it looks like it stores all the Polly MP3s for you locally. Sweet.

 

Custom Alexa Node-Red Skill

Custom Alexa Node-Red Skill

Finally! I am surprised it didn’t even take as long as it usually does for me to figure shit out. Took a few weeks. As usual, I did not come up with the solution on my own but found it out on the web and slapped it all together. I wanted a custom Alexa Node-Red skill, to be able to take a command given to Alexa and read back data from one of my sensors. Things like temperature sensors, water level, etc. I wanted to be able to ask Alexa what the values are. What I got: exactly what I wanted. It all works. There is two parts to this: the Node-Red flow and the Alexa skill.

Alexa Node-Red

First off, to get any of this working you must have your Node-Red server accessible from the outside world. That means port-forwarding, DNS, domains, SSL, all that. It’s fun getting it all working. Not. Just like my previous post, I happened to have it already setup. Once your Node-Red install is available from the web you are good to go. Now you don’t need the entire NR setup opened up either. I just allowed a few NR served pages to be available. Not the entire NR itself.

Update: I made a new post about Node-Red behind a reverse-proxy/SSL

Let’s Begin

It starts with a regular HTTP node to a switch node. That switch node splits up Alexa’s requests to NR; LaunchRequest, IntentRequest, SessionEndedRequest. LaunchRequest gets invoked when the skill starts. You could have Alexa say “Hello what do you want?” for example. IntentRequest is the goods. Then theres SessionEndedRequest, I’m assuming this gets called at the end. Haven’t toyed with it. Then you pass those requests off to do other stuff, like the DoCommand where it grabs your intent? Then a function node to extract the commands, which gets passed off to another switch node to split up the possible commands you can give Alexa. Give her as many commands as you want, then there is a “device doesn’t exist” at the bottom. This is used if she didn’t hear you right or the device doesn’t exist. All that data gets passed to a template to format what Alexa will say and sticks the data in JSON. Bam! That wasn’t so hard right?

Here’s the flow (all standard nodes):

That’s the Node-Red half. You are not done yet. On to the Alexa skills half. This part is easy don’t worry. Login to your Amazon Dashboard and click Alexa. Choose “Get Started” with the Alexa Skills Kit, click add a new skill. Under Skill Information give it a name and choose the invocation word, what you will say to Alexa to start your skill. I chose “Node Red”, so I have to say “Alexa, ask Node Red….”. These can be changed at any time it seems. You won’t be publishing this skill, it stays beta for only you to use. For the Global Fields section, no you will not be using an audio player. Well, maybe you will but I didn’t, and it will probably change things for you.

Note about the flow: The NR flow works (for me) just fine however I noticed it throws an error in the debug tab whenever a command is called. If it is an unrecognized command response it doesn’t throw the error though. It complains about headers already being sent. I will update the flow if I find a fix for it.

Interaction Model

Intent Schema

Intent Schema

This is the part of the Alexa skill where you tell it what to do. It is pretty straight forward. Just copy this to your “Intent Schema”. There are no custom slot types and no values to enter.

Sample Utterances

Sample Utterances

This is where you list the invocation phrases that will activate Alexa. Normally (and in other online tutorials for Alexa skills) this is where you add a ton of different phrases. But we are not. Node-Red is going to handle that side for us. This box just gets one line of text.

Configuration

Global Fields/Endpoint

For a service endpoint you are going to pick “HTTPS”. In a lot of other tutorials you will usually choose AWS Lambda but we are doing all of our own heavy lifting with NR. We don’t need no stinking Lambda. Choose your closest location and enter the URL that your Node-Red is accessible from (via the web remember). Say no to account linking and you can also leave Permissions alone.

SSL Certificate

Certificate for Endpoint

Choose the option that bests describes you. Most likely it will be the first option. For me I am using a subdomain that is already SSL’d with Let’s Encrypt so I choose the second option.

Test

Basically just leave the toggle flipped to enable the skill for you to use. you don’t need to do anything else on this page.

Publishing Information

Nothing to do here, you won’t be publishing this skill. Why? Because it requires too much setup on the users behalf. I don’t think Amazon would approve a half functioning skill that requires advanced user setup to get working. You could always try. Good luck.

Privacy & Compliance

Three no’s and one box to check, I mean as long as it all applies to you right 😉

Done

That should be it. With Node-Red available to the web and the flow implimented and with the new Alexa skill you just made you should be good to. I hope you found this useful, I sure wish I had found a blog post like this. Now go test it out with your Amazon Echo/Dot!

At the time of this writing a beta product appeared in the Amazon Dashboard for a the “Skill Builder”, looks to be a new UI for building Alexa skills. If this gets implemented for everyone in the future things may be different than they are described in this blog post.

This is where I found the goodness, buried deep in comments on a (awesome) blog.

Receiving a Twilio SMS

Receiving a Twilio SMS

 

I use Node-Red for a lot of things and I have recently wanted to send notifications via SMS instead of an email, and I wanted to be able to receive SMS messages as well. I have had issues with the email node. It doesn’t always send for me (I’m using Google’s SMTP) and I don’t have a loud email notification on my phone. Because tons of emails thats why. But text messages on my phone have a very loud notification. I also wanted to be able to send messages to my Node-Red server, doing that via email is cumbersome. So texting it is. Plus sending/receiving an SMS is way cooler than a boring email. I know I can send texts to my phone via email (on most carriers) but that still requires using email and the email node. Don’t want to.

Enter Twilio. I have used Twilio before in the past with their free service tier and never got around to actually using it. The free service lets you do all the same stuff (I think) as the paid service but all the texts have a “This message was sent by Twilio – ” tag attached before every message. So I think thats why I never used it. Well I looked into it again and this time I just upgraded my account right away. Not fucking around. I just threw down $20 bucks. You buy a phone number for $1 per month and then you pay for all the messages you send. The cost per message (at this time) is somewhere around $0.0075 cents! Sweet! I figure I should be able to get a years worth of my number rental and a few messages off that $20 bucks.

So lets use Node-Red to send and receive an SMS with Twilio, it’s why you’re here right? Not for my ramblings.

To send an SMS from Node-Red is dead simple. Just install the Twilio node. It gives you a new output node. Just send you data to the node and off the SMS goes.

To be able to receive a message from Twilio your Node-Red needs to be accessible from the outside world. Twilio needs to be able to see the XML we are going to generate. In my searches for doing this I got a fucking headache. I found a few blogs out there with info on this but nothing concrete. It was actually pretty simple, at least for me. I have also already done this before in a previous post. I happen to run Apache for my web server, which is already setup and open to the world, DNS already setup. All I had to do was create a subdomain (not required) and point a the new vhost to my Node-Red server/install (via ProxyPass). Once this was done Twilio could access my Node-Red served pages. I also have (but I don’t think is required?) SSL already setup. I used Let’s Encrypt on my Apache server. However you do it is fine, you just need to be able to see at least 1-2 Node-Red generated pages from the web. I honestly found this to be one of the more pain in the ass tasks to deal with. Good luck.

Update: added a new post about reverse-proxy/SSL

 

Twilio Flow

The switch node checks msg.Body with   msg.req.query.Body for an incoming command. If the switch matches a command it routes it to a function node. If it matches a command it also checks that command for an ON or OFF and outputs the results as a 1 or 0 for you to do with what you like. So you would text your Twilio number “mySwitch on” or “mySwitch off” and bam! Otherwise it passes it off to a different function node to process the message further. That node checks the incoming message for a global. By this I mean you text your Twilio number “read mySensor” and this function node sees the “read” command and then checks for a sensor name (mySensor). It matches this name against a global you have saved and spits back the results after being XML’d. So when you text “read mySensor” it checks the mySensor global and texts you back with “mySensor: yourvalue”. Awesome.

Tip: the switch node is case sensitive, so be careful. The function nodes change everything to caps I assume to simplify stuff I dunno. I didn’t write it. Also when you change the sensor name make sure to edit the substr to match the character limit of your sensor name. Example: WEMO would be 4, MYSWITCH is 8 characters. Same thing if you change the READ query.

Here’s the flow, I included a testing global for you as well:

This is where I got the information on the Twilio node (from the creator it seems) and how to receive messages. But I found it about as clear as mud. (Note: the json2xml node doesn’t exist anymore. Just use the XML node now.)
http://blog.thiseldo.co.uk/?p=1090

Here is the original flow on flows.node.red.org, but its a little outdated.
http://flows.nodered.org/flow/d65e0c5e4f5fef767be2

 

Some other pages I found helpful along the way:
http://flows.nodered.org/flow/bce112d484f93a8c282a
https://groups.google.com/d/msg/node-red/YBVKXhJLI90/gUh3T-O6BAAJ
https://www.thethingsnetwork.org/forum/t/parse-data-with-node-red/835/9
http://noderedguide.com/node-red-lecture-3-basic-nodes-and-flows/
https://github.com/Leonidas-from-XIV/node-xml2js/issues/87
https://www.twilio.com/docs/api/twiml/sms/your_response

IVR with Twilio and Node-Red example flow
http://flows.nodered.org/flow/637b5f6128a8d423503f