HVAC-Talk: Heating, Air & Refrigeration Discussion banner
1 - 20 of 52 Posts

CrystalPeakSec

· Professional Member
Joined
·
15 Posts
Discussion starter · #1 ·
Hi all!

I'm the founder of Crystal Peak Security, a team of security researchers focused on the hardware/firmware security of Business Automation Systems.

A steady drumbeat of security vulnerabilities are constantly reported weekly, covering a variety of products. With such a high volume of information, it's difficult to determine how Building Automation Systems are affected and how to assess the risk to installations. Each week, we summarize relevant Cybersecurity Advisories published by CISA so you can quickly and easily determine which of the many products at risk apply to you. Much of the information is going to be irrelevant to you, our goal is to make it skimmable in ~30 seconds to easily determine if something you use has an advisory so that it can be addressed.

This week's issue can be read here: crystalpeaksecurity.com/posts/07_03_2023_cisa_summary/
Vendors that are affected:

- Delta Electronics
- Enphase
- Hitachi Energy
- Medtronic
- Mitsubishi Electric
- Ovarro
- Rockwell Automation
- Schneider Electric

Feel free to reach out with any questions or feedback! I'll be posting these summaries weekly. Cheers.
 
I looked at the schneider one on your website. It only had an announcement from Schneider itself, and nothing else. Is that typical?
My experience is BAS manufacturers are generally told about their exploits(do not find it themselves), and publish a bland 'update now' announcement. But they do not give details as to what exactly the vulnerability was, how it was disclosed, how an attacker might exploit it, and other grittier (but less positive for the company) details.
Do you publish stuff like that?
 
Discussion starter · #3 ·
My experience is BAS manufacturers are generally told about their exploits (do not find it themselves)
It could happen either way. Often times manufacturers have internal teams looking for flaws. Issues found internally are still assigned a public CVE number and an advisory is put out once the issue is patched. When things are reported externally, the researcher will typically give a disclosure timeline which puts a lot of pressure on the manufacturer to fix the issue fast so that it's patched prior to disclosure. This incentives them to discover the issues internally so that they have a more favorable disclosure process and timeline.

The CVE will typically contain a decent amount of detail regarding what the issue is, when it could manifest, and what the impact would be if exploited.

Do you publish stuff like that?
We don't typically dive into the low level details unless we discovered the issue ourselves (we'll be publishing some work on a BACnet implementation soon!). We do sometimes have customers that ask us for that additional context around a public vulnerability to determine if they are at risk, we'll dive into the low level details (sometimes reverse engineering the firmware/product) to see if it's exploitable in the way they're using the product.
 
It could happen either way.

The CVE will typically contain a decent amount of detail regarding what the issue is, when it could manifest, and what the impact would be if exploited.
Of course it can happen either way.

Do you link to the public CVEs? Or will you start?

Oh wait - the colored text of the manufacturers named software isn't a link to that manufacturer or their software, it is a Link to the CISA report on the software. Oh that is not straightforward, but once you know that it is aright.
 
It only had an announcement from Schneider itself, and nothing else.
SE is really good on announcing them. If they issue a CVE, they send out a notice in short order. They have a mailing list for them that seems to cover all their products. From UPS, BMS, PLC, etc. You might need a SE login, but they typically link to more details for each issue. Its obviously not the here's how to exploit this issue level of detail. Think CISA has a OT specific mailing list that boils things down to OT stuff. BMS typically land in there as well as the full list. If I see SE product land on CISA's mailing, I either already got notice or do within a few days from SE.

https://www.cisa.gov/topics/industrial-control-systems
 
Its obviously not the here's how to exploit this issue level of detail.
Disappointing. That level of detail can be informative AND entertaining. Here is a talk at Defcon 27 on hacking a Delta Controller.

https://youtu.be/uJP061PUxgY

Unfortunate cover on the video - hopefully I don't cross a mod somewhere....
A fun before screenshot:
Image

And a pair of after screenshots. Hilarious.
Image
Image
 
Are you trying to restrict the information BAS or are you also targeting other customers? Because thread title here suggests BAS and I'm seeing things that I would consider PLC or DCIM or even power grid worlds in the example page you posted. When I look at the example page I'm seeing one thing at the very bottom that would interest this audience. We are a small subset of a much larger industry and I'm already getting CISA Vulnerability Summary reports that I search for some key words and it's very often I find nothing of interest to ME or even my community.

Something focused on our market would be nice.
 
Discussion starter · #9 ·
Are you trying to restrict the information BAS or are you also targeting other customers? Because thread title here suggests BAS and I'm seeing things that I would consider PLC or DCIM or even power grid worlds in the example page you posted. When I look at the example page I'm seeing one thing at the very bottom that would interest this audience. We are a small subset of a much larger industry and I'm already getting CISA Vulnerability Summary reports that I search for some key words and it's very often I find nothing of interest to ME or even my community.
We are trying to cater everything we do to the BAS industry, but like you noted, everything BAS related gets lumped into the ICS category as far as CISA is concerned.

We considered filtering out the non-BAS related stuff, but didn't want to accidentally exclude something very relevant. Instead, our goal to start is for it to be skimmable, making it very easy to see if a relevant product/manufacturer has an advisory put out. But as we become more familiar with all the moving pieces in the BAS world, I expect our content to become more specified and original. Eventually, you should be seeing CVE's from us make their way into CISA advisories!

Something focused on our market would be nice.
This is our goal. Are there certain types of information or content that you wish existed?
 
From this issue, Delta and Schneider are the only 2 BAS front end vendors.
BAS being a Building Automation System - traditionally encompassing HVAC and Lighting Controls, but can include a variety of other systems on the OT network.

Enphase = PV inverters, EV chargers.
Hitachi = Maybe VRF systems, haven't see much of this in my market.
Medtronic = Medical devices
Mitsubishi = Usually VRF systems or Elevators
Ovarro = Process Industry, Oil and Gas, Water, Leak Detection
Rockwell = Industrial/Factory Automation

Here's a list of vendors having any IP addressable device(s) that you should make sure is on your radar. There may be more.
AES Corp (Fire Products)
Alerton (Honeywell)
Automated Logic Corporation
Belimo
Bosch
Cimetrics
Contemporary Controls
Cradlepoint
Daikin
Delta Controls
Distech Controls (Acuity Brands)
Eaton
Encelium
Enlighted
EnOcean (Lonworks)
Galagher (Access Controls)
Honeywell
Johnson Controls
Loytec Americas
Lutron
Lynxspring
Mitsubishi Electric
MOXA
Optigo Networks
OSRAM Sylvania
Reliable Controls
Tridium Niagara (multiple brands)
Schneider Electric
Siemens
Trane
Trend
Wattstopper (Legrand)
 
Discussion starter · #12 ·
Thanks for the list!! I'm very interested in MS/TP reachable surfaces as well, they seem to be very under looked.

We struggled to draw a boundary around BAS because of the "...can include a variety of other systems on the OT network." I'd imagine a BA controller sometimes interfaces with factory automation devices from Rockwell? Or elevator devices from Mitsubishi? I'm wondering if BAS shops handle other parts of the automation network for their customers or if they stick strictly to BAS.

Either way, I'm hearing that some additional filtering is in order. Thanks everyone.
 
MS/TP reachable surfaces as well, they seem to be very under looked.
Because that would require physical access to a local communication buss that is not generally exposed to occupied space. Besides that there are a number of sensor busses that do extend to occupied space that are worth looking into as they can grant access to the entire system.

I'd imagine a BA controller sometimes interfaces with factory automation devices from Rockwell? Or elevator devices from Mitsubishi? I'm wondering if BAS shops handle other parts of the automation network for their customers or if they stick strictly to BAS.
Yes BAS can integrate to those systems but rarely will a BAS shop also take on support for those systems.
 
I'm very interested in MS/TP reachable surfaces as well, they seem to be very under looked.
Given that BACnet/Ethernet and BACnet/IP both send data as cleartext, I'd call them unsecured by default. Should be isolated anyway. Any MS/TP device behind a BACnet router follows.
BACnet S/C adoption will be slow.
if you mean MS/TP being an attack vector, that would fall under physical security - i.e. someone with a serial based device physically plugged in to both the serial and IP network.
 
You can click through to view the CISA advisory to view details of each vulnerability. The nature of the vulnerability drives the CVSS severity score; easy to exploit with high impact gets a Critical score, hard to exploit with minimal impact gets a Low score.

That's detailed in the "Vulnerability Overview" section of the CISA advisory.

For the Schneider advisory, that section includes:

--- begin ---

3.2 VULNERABILITY OVERVIEW

3.2.1 IMPROPER CONTROL OF GENERATION OF CODE ('CODE INJECTION') CWE-94

Schneider Electric EcoStruxure operator Terminal Expert versions 3.3 SP1 and prior are vulnerable to a code injection attack that could allow an attacker to execute arbitrary code and gain access to all information on the machine.

CVE-2023-1049 has been assigned to this vulnerability. A CVSS v3 base score of 7.8 has been calculated; the CVSS vector string is (AV:L/AC:L/PR:N/UI:R/S:U/C:H/I:H/A:H).

--- end ---

Manufacturer's security advisories sometimes provide additional detail, but only the biggest manufacturers seem to have security teams that publish their own, more detailed security advisories. Some manufacturers do not provide security advisories at all.

Schneider's security advisory for that vulnerability provides additional detail, e.g., "a local user of the Windows engineering workstation" could exploit this vulnerability to cause "unauthorized code execution" that could "result in loss of availability, confidentiality, and integrity of the workstation".

The CISA advisory also discloses who found and reported the vulnerability, in section 3.4 RESEARCHER.

Over time, we can see whether the manufacturer's security team is finding vulnerabilities in their products themselves, or if independent security researchers are finding them.

(Full disclosure: I work for Crystal Peak and prepare our weekly vulnerability summaries.)
 
Aright crystal peak guys. Seeing as you all keep multiplying, can you please answer a few questions?

I see the blog on your site goes back to June '23. Is that when you all opened shop? Got any history there? I am not looking for a 90 minute documentary but something that tells us about who and what you are would be swell.

You pentest BAS stuff? So you want to work with OEMs? Like, say TRANE, and then reverse engineer their stuff, and show them where it is vulnerable and suggest fixes for a fee? More or less?

You say there on your site that you all are experienced, but here we are listing who your customers are. So experienced with what exactly? It is not with BAS stuff. Not trying to be mean, just following the logic. Asking for an answer, not throwing rocks.

Do you work with integrators like most of us? Not TRANE the OEM, but the poor schmuck in Super Cool Automation in Anytown USA who is sitting on a site with a bunch of TRANE stuff trying to get it to talk. There are a few OEMs represented on here, but not too many.

I appreciate you spending time with us. Informative. But how does your posting here turn into dollars. I think I know too little about your gig to understand....

I may have more questions, but this is a good start.
 
Discussion starter · #17 ·
I see the blog on your site goes back to June '23. Is that when you all opened shop? Got any history there? I am not looking for a 90 minute documentary but something that tells us about who and what you are would be swell.
Sure! My name is Anthony, my background is in low level security research. All my professional experience is doing more or less what you described: Reverse engineer something, find vulnerabilities in it, help make it less vulnerable.

Crystal Peak has been around as a small single-person (me) consultancy since about June 2022. Around April 2023 we put up a simple website, and those posts in June are our first attempt at creating content for the BA industry. BA became our specialization of choice because I don't think it gets the security research attention that it deserves and it carries a lot of real-world impact with it.

Do you work with integrators like most of us? Not TRANE the OEM, but the poor schmuck in Super Cool Automation in Anytown USA who is sitting on a site with a bunch of TRANE stuff trying to get it to talk. There are a few OEMs represented on here, but not too many.
I think our skill sets are most valuable to the OEMs - our bread N' butter being the analysis of hardware/firmware & source code. However, we do a ton of relevant software development to support that stuff which might be really useful for integrators. I'm listening for the problems that schmuck in Anytown frequently encounters to understand what help we can offer.

I appreciate you spending time with us. Informative. But how does your posting here turn into dollars. I think I know too little about your gig to understand....
I'm just excited to see folks reading our stuff and giving us feedback. I don't think it turns into dollars on it's own, I'm more here to learn about the industry, help out however I can, and get our name out there. If someone finds our input valuable and wants to engage however, I certainly wouldn't turn it away. Ideally that someone would have some hardware/firmware/source code for us to rip apart, find vulnerabilities in, and fix. But we could do custom tools as well.


feel free to fire off any other questions!
 
Here's a list of vendors having any IP addressable device(s) that you should make sure is on your radar. There may be more.
AES Corp (Fire Products)
Alerton (Honeywell)
Automated Logic Corporation
Belimo
Bosch
Cimetrics
Contemporary Controls
Cradlepoint
Daikin
Delta Controls
Distech Controls (Acuity Brands)
Eaton
Encelium
Enlighted
EnOcean (Lonworks)
Galagher (Access Controls)
Honeywell
Johnson Controls
Loytec Americas
Lutron
Lynxspring
Mitsubishi Electric
MOXA
Optigo Networks
OSRAM Sylvania
Reliable Controls
Tridium Niagara (multiple brands)
Schneider Electric
Siemens
Trane
Trend
Wattstopper (Legrand)
How could you miss Temco / Bravo controls from the list? Won't be shocked if that's an epic train wreck.

Certainly more, few that come to mind.

Liebert / Vertiv or whatever the name of the week is.
Stulz
SquareD
Cyberpower
Really anyone making data center cooling / power solutions. Gensets, PDUs, UPS, etc. Even just power meters with IP are all over now.

VFDs these days are starting to pop up with IP
ABB
Danvos
SD
SE
Siemens

Then there is a ton of IP gateway to xyz vendors.
Fieldserver - did they get sold to MSA?
Babble buster

Might as well add networking gear as well. Especially some of the common stuff you see in BMS.
ccontrols.com
SOHO junk from big box stores

Don't some of the Trane package units with the new symbio controls come IP now? Thought I have seen a few recently. Certainly have used ones that the usb interface shows up as a NIC.

There is a ton of IP gear out there in typical BMS and a fair deal of it really prob shouldn't exist.
 
Figured there would be more, my list is just a sample of device manufacturers that appear in my realm.
VFDs are still Lon or BACnet MS/TP over here.

Sent from my Pixel 4a using Tapatalk
 
Can't say I have seen a native lon VFD in a twenty years or more on a new install. Not sure what brand it was I ran into recently, but IP was standard. Its optional on a many now. Vacon through a channel we offer had an IP option card back in the mid 2000s.

https://partners.trendcontrols.com/trendproducts/cd/pl/pdf/en-ta200826-uk0yr1207.pdf

Your still ordering VFDs that come native lon or is it an option card?

Humidifiers now that I think about it. Its either an option or standard on dristeem humidifiers.

Autoflame DTIs boiler controls, IP is standard.

Emerson asset monitors, aka fancy vibration gear, IP standard.

I may even have some RO/DI skids with IP. There is a ton of stuff that touches our industry that offers IP interfaces and some of these small vendors are almost certainly security dumpster fires. Customer asked for IP interface and we tossed it together. It works, you didn't ask for security...
 
1 - 20 of 52 Posts
You have insufficient privileges to reply here.