Server Rooms, (H)PC and 19" Infrastructure

Head-, storage- and compute-nodes are located in room 5018 in Cauerstr. 1. One key is with Jens, one in office 5007; the reception can always open the door.

Room 5018 (area: 22.74 m² per floor plan) is supplied with cool water from the central cooling system with pipes of normed size DN80. The cooling system (supposedly) provides up to 60 l/min of cool water at 10°C with a sustained ΔT of 9 K. The 10°C should be configurable. This should provide cooling capacity of 40kW.

The room is supplied with electricity from redundant high current sockets of two kinds: 9× 32A (~7kW, red), 6× 16A (~3.5kw, blue). Totalling at 84kW (or 42kW w/ redundancy).

The floor in the room can sustain an area load of 8kN/m² or 800kg/m². With a size of 0,8m × 1,2m the area under one Rack can sustain 768kg. The Rack itself weighs 113kg which leaves 655kg for components.

Contacts: Peter Burzler (p.burzler@fz-juelich.de, 0175 1179338)

Anomalies in the operation (like reduced cooling capacity) do happen, they deserve their own page.

see server-layout

There is one switch in each rack for each boot, IPMI and InfiniBand network. Each switch of one rack is connected to it's counterpart in the other rack.

The cable bridge across the ceiling provides 12 LAN cables from the JuNet switches. They are color coded by function:

• 4×yellow for PDUs,
• 2×green for the DCMs,
• 1×blue for the LCP and
• 5×black (10GbE) for the Ethernet traffic of servers (hn-1, sn-1, hn-1-ipmi, wind, fire).

A 10G switch module to provide 10G connectivity for the servers, has arrived from the Network Department (JSC, Herr Frank Mohr) in May, 2022. (bottom most slot 7 of the the NET354 switch casing in the network rack).

Each Rack has

• 2 Power Distribution Units (PDUs) which redundant supply power for the components
• 2 Power Extension Banks
• 1 Door Control Module which opens the magnetically closed doors when a temperature threshold is reached

The Power Extension Banks are needed, because the PDUs have 1 16A fuse for all the C13 sockets and 1 16A fuse for 2 C19 sockets. The Power Extension Banks for the 2nd Rack are actually full blown PDUs with a network interface which currently is not connected.

The threshold for the DCM can be set in its Management interface. For the URL see the table below.

The “Intel” rack has two 19“ PDUs, that have been moved from the Nürnberg rack to Erlangen in 2021. They currently do not have a sensible (134.94.72.0/23) IP associated with their JuNet IDs.

It has one server rack with LCP. It is currently (as of late 2022) unused, empty, and disconnected from the mains electricity.

The “rack plan” is arguably anticlimatic.

The server room has three 4.5 kW AC units (Umluftkühlgerät, ULK). They are controlled by a controlling device in the room.

The rack has two redundant power distribution units (PDUs, Schneider Electric AP8653 max. 7360W):

1. http://pdu1.iek.kfa-juelich.de (left in rack)
2. http://pdu2.iek.kfa-juelich.de (right in rack)

The contact with the landlord for the cooling units is Jürgen Kunz (jk@ecg-fm.de, 0911 321 599 82).