Ch.1: The dilemma.

In the summer of 2017, National Nursing and Rehab (NNR), my dad’s home health company, had a dilemma on its hands. The company stored most of their patient information “in the cloud” with a company know as Home-care Home-base (HCHB). In laymen’s terms, HCHB was a google drive for nurses that charged $1000/gigabyte/month. Smart, huh. Compare that to Apple’s iCloud Drive or Google Drive storage of 1000 gigabytes/$10/month. NNR had about 80 gigabytes of patient information. Do the math… Ouch… An alternative had to be found.

Ch.2: The decision.

Dad came to me and presented the problem. I suggested we do away with the 3rd party cloud service of HCHB and host the files from an in-house file server. The decision came down to either a Dell rack with Windows Server (~$1,000 one time cost + negligible monthly electric bills) or HCHB’s ~$80,000/month for the foreseeable future. The former option was chosen.

Ch. 3: April Anthony vs. the IT world.

[Backstory: In the late nineties April Anthony (AA) thought it would be a good idea to sell cloud storage at an extremely marked up price to healthcare providers. These providers (like my dad) more often than not happened to be uneducated in cloud storage and therefore initially believed they were getting a good deal. They had been sucked in. AA had some IT guys make the software from her idea, she marketed it and became a millionaire overnight. Thus HCHB was formed. AA now glides around the country in a Learjet. Good job AA.]

Back to the narrative…

When asked for our files HCHB simply said “no”. They did not want NNR to leave their system (I mean, who wouldn’t want a client like NNR to leave. I’m sure AA likes getting a monthly check for 80k.) In an initial conference call NNR gave HCHB that we were abandoning use of their service asap. HCHB essentially told us good luck surviving with out them, and good luck getting our files off of there servers. We still had access to all the files in the cloud, but without the help of HCHB the only way of obtaining copies of all the files was to open>print>save as PDF. One by one, until all files had been saved. There were about 31,000 medical records and 60,000 patient attachments (mostly PDFs). Challenge accepted!

Ch. 4: The Veruca Salt approach

Remember in Charlie in the Chocolate factory when Veruca Salt’s dad had his nut cracking workers unwrapping thousands of boxes of Wonka chocolate instead of their usual nut cracking job? Similar to that, NNR decided to attack the file problem with as many hands as possible. All through May they paid employees overtime to work on the ‘HCHB Project’. Employees would sit at their desktop and save files one by one, for hours on end, barely making a dent in the ~90,000 file tall mountain. They had gotten through 2000 files in May alone. At 2000 files a month it would’ve taken NNR almost four years to save all the files! In June NNR decided to hire some of the managers’ college-age children as summer interns to work on the project and hopefully speed things up. June is when I was first briefed on the project.

Ch. 5: Manwholikespie

I argued that the current way of saving files was very inefficient. Dad asked how I’d otherwise get the files. I knew other ways were possible, but I had no idea where to start. When I was officially asked to work on the HCHB project I agreed on one condition: NNR also hire my friend from high school- Robert. Robert is currently attending Texas A&M University with a major in computer science, I believed if anyone could make a program to speed things up it would be Robert.

Ch. 6: The Beginning of the Heist

It was more probing and exploring how HCHB works the first week Robert and I sat down to work on the project. Neither of us had seen an application in the format HCHB was designed in. Our initial thoughts were that HCHB would use a file server, sftp or the like… instead HCHB uses overly complex Citrix virtualization, similar to VMware, Citrix is a virtual machine environment mostly aimed at single use applications, or desktop virtualization for secure remote access. Through our study of HCHB looking through the lens of Citrix, we discovered that the HCHB application was emulated on individual user accounts on a standalone Windows Server 2008 image. Each nurse had a HCHB login, which corresponded to a user account on the Server 2008 image. Citirx receiver is required to login to HCHB. Citrix receiver is a client app and essentially a Remote Desktop client, that limits use to one specific application.

So the application user flow would be Nurse’s Laptop->Citrix Receiver Client->World Wide Web-> Citrix Zen Server-> Windows Server 2008 VM Image->Citrix Zen App-> Actual HCHB desktop application.

HCHB had Citrix so locked down that while in the Remote Desktop interface, one could reposition windows on the desktop, but there are no shortcuts, start menu, or windows file explorer to be found.

This limited desktop can be accessed through Citrix Receiver (MacOS client only, It is impossible to get to this desktop view on the Windows Client App.) Menu-> Edit->Send Ctrl. Alt. Delete->{opening task manager from the windows “Ctrl. alt. Delete” menu that appears}

Ch. 7: The Heist

Known factors: HCHB runs on very secure a virtualized Windows server.

Unknown factors: how we would get our files.

At this point Robert and I realized our dreams of simply copying and pasting all 90,000 files in one go were dead. We needed to find another method. We each attempted a plan to capture the files.

I thought it would be possible to upload the files directly to google drive from the server 2008 image if I could get access to a web browser/file-explorer on the remote VM’s desktop. Playing with HCHB’s app, I discovered one could open a limited windows file explorer, but only to open PDF files in Adobe Reader 9. Once in Adobe Reader, I could open the online help window. That opened up Internet Explorer to the Adobe Community forms. However, the internet explorer window was limited to the Adobe Help forms and did not allow you to type in a URL such as google.com. I created an Adobe support form titled “Is it possible to hack HCHB through adobe reader.” One clueless form member tried to write a helpful answer, but that was not what I was after. Back on my laptop I returned to the form I had created and pasted a comment with a simple google.com link. Back on the 2008 server VM, I returned to the form through Adobe Reader’s basic Help Internet Explorer window, navigated to the form I had created, clicked the Google link, and boom. Access to the entire internet through Adobe’s support forms, through Adobe Reader 9, through the locked-down HCHB client app. Now what? I navigated from google to download and install Classic Shell on the 2008 server. Classic Shell is an alternative to the windows Startmenu/file explorer. I now had a desktop, “start menu”/taskbar, and a file explorer. I though I had it solved. I figured I would open the folder with all of NNR’s files and upload them to my google drive and we would be done. Wrong. HCHB has cleverly stored all the files in an sql database. We didn’t have a sql server, nor know hot to operate one. Shoot. Dead end.

On the other hand, Robert had chosen a wiser route and begun working on a script to automate/speedup the process of saving files one by one. Robert used python as a base and chiefly autogui as a means of locating the buttons in the GUI that a nurse using HCHB would see and use. Robert’s program would accomplish the same task as if a person were sitting in-front of the computer, selecting a file, print, save as PDF, and then selecting a file directory to save it into. There were some odd hurdles to overcome, such as waiting for the “save file” window to appear. Sometimes it would appear instantly, other times it would take a couple minutes. We think this unpredictable timing was a result of file size and network speed to assemble the PDF from HCHB’s sql database. The second big hurtle was a result of a glitch in Citrix’s client app. Sometime a document would save as intended to our local machine, other times Citrix would save the file to the home user’s documents folder of the Server 2008’s logged in user. We found no explanation for this. The third hurdle was the use of nurses’ HCHB accounts in general. One account could only be opened on two machines at a time, and must be logged out when not in use. If left logged in, a newly created remote instance may pickup where the last ended, vice versa, or result in no access all together. I think we needed up using 3 or 4 different nurse logins to complete the project. The last hurdle was the fact that Robert’s program could only run one instance at a time per local machine. To overcome this, we ran the program on as many physical desktops, laptops, and virtual machines we could reasonably utilize for the project. Keep in mind, the client application, setting and method we were using required we use MacOS as our client operating system. Robert’s program could be adapted to Windows, but we opted not too.

We kept track of what files had been downloaded and what we had left to save by office branch and what computer saved the files via this Google Doc Spreadsheet.

Ch. 8: Tying Up Loose Ends

We ran Robert’s program over the course of a week and managed to save all of the medical records. We still had no way of getting at the patient attachments. Keep in mind the medical records were plain text information populated into PDFs, whereas the patient attachments were JEPGs or PDFs scanned in and uploaded by nurses. These files were stored on HCHB’s network attached storage, not in the 2008 virtual image, out of our reach. The only way to get these files was to either break into HCHB’s headquarters or get them to comply and hand over the files to us. We choose the latter before pursuing other action…HCHB agreed to give us an SFTP login to their file server for two days… for the small price of $2000. Problem solved. We had the patient attachments downloaded within the first hour.

Ch. 9: Case closed

In the end, if my math is correct, our project saved NNR over 3 million dollars. I come to that conclusion by assuming had NNR completed the project using the Veruca Salt approach, they would be paying $80,000/month for HCHB file storage until the team had saved all the files. Recall it took the team a month to get 2,000 files. At 2,000 files/month of 90,000 files would take the team about 4 years to accomplish the task. Four years of HCHB monty payments would equate to $3,840,000 ($80,000 x 12months x 4 years). Not to mention paying the salaries of those 9to5 employees working on the project. Our files are now securely stored on an in-house HIPPA compliant file server.

 

(If you are one of the few remaining companies who work with HCHB and would like to move to a HIPPA compliant *archive-based* in-house system, contact us!)

*Archive-based means we store files for the purposes of simply securely archiving them incase your local friendly Medicare/Medicaid auditors want to see them: not a database for routine access by active duty RNs. For that look into using KanTime.