One of the new features of Ipswitch's MOVEit Automation 2018 product is the introduction of REST APIs. In this video from Ipswitch contributor Adam Bertram, we'll cover how to use PowerShell to first authenticate to a MOVEit Transfer server and then we'll cover how to create and monitor a file transfer job using a PowerShell script.
In this clip from TechSnips, you'll learn how to use MOVEit Automation's new REST API to authenticate to a MOVEit Transfer server and create and monitor a file transfer job with a PowerShell script. For more information, including script excerpts, check out our previous posts on authenticating MOVEit with PowerShell and creating and transferring files.
Big thanks to Adam Bertram's awesome new site, TechSnips, for the video. TechSnips is a new (and totally free) is an IT career development platform that offers video how-to's on dozens of IT technologies and techniques. Adam and co. have covered tons of technologies, like cloud computing, programming, system administration, and a whole lot more. Check it out!
Adam Bertram is a 20-year veteran of IT. He’s an automation engineer, blogger, consultant, freelance writer, Pluralsight course author, and a Microsoft Cloud and Datacenter Management MVP. Adam focuses on DevOps, system management, and automation technologies as well as various cloud platforms mostly in the Microsoft space. Catch up on Adam’s articles at adamtheautomator.com or follow him on GitHub or Twitter at @adbertram.
In this next video we're going to talk about using Ipswitch's MOVEit Automation 2018 product and its new REST API is that it's got, and we're going to use those REST APIs to show you how you can initiate programmatically in a Powershell script to transfer files from one source to a destination.
So, before we do that there's a few prereqs that we're going to need to talk about. Obviously, you need MOVEit Transfer 2018 and you need permissions to do that in MOVEit Transfer.
Obviously, I'm using the admin account.
There are a few things I've already done. I've already added files manually in the source folder into Transfer and you need to just make sure that that's also accessible.
To explain this, the scenario is we have a file called wallpaper.png and it's already uploaded into the script like I showed you. So here in MOVEit Transfer, in our source folder here we have a file called wallpaper.png. So I've already done that ahead of time, so let me make sure, here's the destination folder we're going to be using and I've already had that if there for my testing so I'll go ahead and just delete the other UI here.
Alright, so now that we have that ready to go and let's see how we can transfer that to the destination folder. The reference is here, this is a really good reference link if you want to get more information about copying files with the API. This gets pretty code heavy but I will show you here at the bottom that a really shortcut on how to to get this done quickly. Alright so, first off the first thing we need to do we need to craft and endpoint URI so in a REST API everything has an endpoint and you need to have the URI to point your script to your code so on the first few lines here I'm just defining some variables the localhost is going to be locally on the MOVEit Transfer server itself and I'm spying the MOVEit Transfer username and password and then grant type that just means normally you can authenticate with to REST API with a password or with the token. I'm just going to be using a username and password here.
And then on line 28 that's where I define the endpoint URL so it's going to be “localhost/api/view1/token”. That actually gets the endpoint URI to grab the token. We have to grab a token first then once we have the token we can use that to authenticate. I’ll show you how to do that. And then on line 29 there we have the HTTP body which is essentially the HTTP query the git, we’re going to be sending a git verb to the the REST API and that just tells it the grant type the username and password—it provides all that information.
Okay, then on line 31 here's where we actually are going to send the HTTP POST request to MOVEit, so first off in 32 that is a PowerShell specific thing, if you're using something like Python or something else you may not have to do this, but that certificate validation callback method that we're calling there that’s so we don't get prompted for a self-signed certificate so we have a self-signed certificate on the MOVEit Transfer server.
Next up is where we actually run Invoke-RestMethod we pass in the URI we're going to say we do want to send a POST method and then we send the body along with that and that actually grabs the token. So once we have the token then we can use that for all subsequent calls to it so that it knows who we are, so we don't have to keep passing the username and password all the time. All right so I will go ahead and run this. and then hopefully—yup, we didn't get an error message which is good—and now we have a big long gobblitygook encrypted token that we can use to authenticate ourselves to the lua transfer server.
All right next up, we need to prep the headers here and what I'm doing here is I'm just essentially creating a headers value that I'm going to pass later. I'm going to use the authorization key here and pass it the value of the bearer token that's how Invoke-RestMethod actually can then pass that information to the MOVEit Transfer server. Once we have the token, and that's all that is done then we need to find the file ID. MOVEit Transfer depends on file IDs quite a bit so the first step we need to find the file ID, that wallpaper.png file that I showed you before.
So to do that we need to query the files API which we do that by defining the endpoint URL notice that instead of token it's files now and then on line 43 here that's where we actually can query that out and pull that individual file out. And now you can see that we have ID, name, path, we have all kinds of information about it. So now that we have the file object created now we need to find the destination folder ID and to do that, same kind of concept here we use the import URL, this time it's folders instead of files and we’re doing the same approach. I'm just looking for the folder name of the destination folder and when I do that notice that now that I have a folder object created to have the ID parentId all that good information there.
All right next step: now we can actually do the copy. So the copy how I'm doing this is I'm creating a hashtable in 52 called folder and it has a destination folder ID key in it and using the MOVEit Transfer folder ID inside of that and then on 55 is where I’m converting that to JSON. I'm doing that because online 57 there you see the body parameter we have to have JSON and or you can just create JSON on the fly if you want but I found that in PowerShell I could just create a hashtable, pass it to convert to JSON on 55 here and then it just creates that JSON for me so that the copy method on 56 here and the URI that we're going to be using it requires that JSON. So I will go ahead and run this and everything is pretty much the same but instead of URI we're passing the same URI there the body like we just talked about and the content type is necessary we have to sys by application/json because if we don't it's going to be an undefined mime type and it's just not going to know what kind of format the call was supposed to be in.
So now we run this. Notice it's pretty quick, it's only a seven-megabyte file and it's all locally but what we did was we actually just copied the file it was just that easy. Everything is pretty quick don't you get all the code built out. Building out all the code is the hard part but I've done all that for you and if you're doing this with PowerShell I'm going to show you a really easy way to do that to sum this all up.
But just to make sure that this thing is in the destination folder now. Tada! Yeah it is. So that actually does the transfer. Let's delete this out and I'll show you how we can sum all this up together so you don't have to worry about all that code if you don't want to so.
So I've created a parish shell function here called copy MOVEit Transfer file and inside of that, it's essentially just combining all that code that we just built together. I have a bunch of parameters: the source file, the destination folder so that you don't even have to mess with IDs or anything like that. You just specify source file, destination folder, username, password and then you know I'm has specify this server name you can set a default value like I'm doing here. And then the rest of this is essentially doing exactly the same thing: creating, getting that authentication token, prepping the headers, finding the file ID, making sure the destination folder is there, getting the folder ID and then doing the file copy.
For the most part it's exactly the same thing, but notice that I've combined it all into one function. I don't have to execute these line by line so in the real world you're probably going to do this a lot of times, so it's a good idea to create a tool to do this so let me bring this in. So now I have copy MOVEit Transfer file, and here I'm going to pass some parameters via PowerShell splatting. The source file path as just as we saw is “/Home/techsnips/SourceFolder/wallpaper.png”. I'm going to transfer it to the destination folder and then there's the username and password that I'm going to use. So I can just copy and bring all that in and notice that we did the exact same thing, I didn't have to go down through each of those line by line. The copy MOVEit Transfer file function does all that stuff for us. And then to prove that it's there I'll go ahead and run it again, and now you can say that a file name with a wallpaper.png already exists in the folder so it did transfer.
So that's a great thing about PowerShell—it's important to learn how the all the code works—but as soon as you learn all that, you can wrap that all up in a single function and between all of the code that we had over here. Notice that we had all the way from 22 all the way down to what 57—37 lines of code we were able to get that we able to do all that same functionality with just, what is that 126-132? With just six lines of code. So that’s a really great thing about PowerShell. So that has been how to transfer files with the REST API using Ipswitch’s MOVEit Transfer 2018.