It seems that just about everything is networkable nowadays.
Thermostats, lights, and refrigerators are now all considered IoT (“Internet of Things”) devices, and someone somewhere probably has a pillow or shoe rack or something that’s also able to connect to wifi.
In the church Production world, it has become similar. Devices that previously had little to no need to be on a network, like video cameras and wireless mic receivers, now can be networked easily, not to mention things like video switchers, lighting consoles, and IEM systems.
Getting an AV device online for management, monitoring, or operation has never been more ubiquitous, and it’s never been easier.
And in many ministries, it’s now common for a production team to have at least some oversight of IT or network operations. So while IT and Production once were two separate disciplines, the lines have now blurred greatly, deepening the expectations that each side have a greater understanding of how the other works.
It’s probably not best for IT to push a firmware update to a switch on Saturday night…
Historically, these are two ministry areas that could regularly butt heads with each other in a fight for control or management of things like check-in systems, online streaming, and even wifi. But in a world of overlapping or converged networking requirements, it’s now more critical than ever that the two sides not only communicate effectively but also work collaboratively to ensure that a ministry’s network is optimized for both AV and standard IT requirements.
“Anytime healthy collaboration is happening between teams, everyone wins,” says Matt Glover, the Director of Technology at Birmingham, Ala.’s Church at Brook Hills. “Production and IT are no different. It’s in the best interest of the Church when these two teams are functioning well together, not just in terms of cost, but in the overall culture of both teams.”
"Open communications will always win the day," shares Keith Brown, Director of Broadcast Ministries, Second Baptist Church, Houston, Texas.
Keith Brown has been the Director of Broadcast Ministries for Houston megachurch Second Baptist Church for nearly two decades. “There needs to be a mutual trust and respect for each other’s discipline,” he advises. Regular meetings need to happen between IT and AV. We have a monthly standing meeting with AV and IT. We discuss ongoing projects on both sides of the house as well as any issues that have come up in the past month. It’s good accountability for both departments.”
Just as regular, healthy communication and accountability are important between Production and Worship or Creative teams, it’s no different with IT, because this is now an area where a simple misunderstanding has the ability to completely cripple a worship service.
“Open communication will always win the day,” Brown says.” There have been times in our organization when a particular production service has adversely affected the overall network. IT educated the AV team and offered comparable alternatives.”
"I have many stories from IT peers about wireless handheld cams showing up unannounced causing all sorts of wifi connectivity ‘fun,’" says Jason Powell, Director of TechOps, Granger Community Church, Granger, Indiana.
Jason Powell has been the Director of TechOps at Indiana multisite Granger Community Church for 21 years and has seen both the highs and lows of the two teams working (or not working) together.
“Both teams are problem solvers,” he explains. “Both teams want the best for their organization. Both teams just gotta get stuff done. However, at times Production may not see the big-picture impact of a technology decision or tool. This seems most common when it comes to wireless AV gear. I have many stories from IT peers about wireless handheld cams showing up unannounced causing all sorts of wifi connectivity ‘fun,’" he adds.
Twenty years ago, the IT lift to support Production applications was negligible. Kids check-in systems would require internet, and maybe some sort of file sharing application to move basic content around the network so it could be shown on screen during service. The pastor may have had a printer in his Green Room or office to print out notes, but that was maybe about it on a weekend.
“…in most cases, AVL traffic should get priority over traditional IT type traffic.”—Jason Powell, Director of TechOps at Indiana’s Granger Community Church
But then came multisite delivery and online streaming. Wireless cameras and IEMs adds extra traffic to the RF space and started to interfere with network access points. Wifi intercom, tablet-based mixing and PA steering, sACN and ArtNet to manage lighting fixtures, and a plethora of Dante-enabled audio devices are all now common at churches of varying shapes and sizes.
It could be argued that on a typical weekend, the network’s priority is now no longer to manage printers and staff workstations, but to support AV applications first and foremost.
QoS (quality of service) is the term for prioritizing network traffic to ensure that the most critical pieces of gear get first dibs on bandwidth, with other things falling in line behind. As the pull on a network increases on the AV side, it’s even more important that conversations about QoS are happening behind the scenes with the IT team so that the important tasks are never compromised by others.
“Adhering to manufacturers recommendations for setting QoS is best practice,” says Brook Hills’ Glover. “This is particularly important for certain devices and protocols that don’t tolerate high latency or jitter [like audio applications or live streaming].”
“In the world of IT, we want the network to be fast and efficient,” Granger’s Powell notes, “but our QoS needs are drastically less than what AVL traffic often requires. An AVL device might constantly run at full line rate whereas most IT traffic is very bursty. If a video call has some dropped frames nobody really cares; however, any dropped frames in a FoH feed is not acceptable. Therefore, in most cases, AVL traffic should get priority over traditional IT type traffic.”
It behooves modern Production directors to educate themselves about IT terms and principles and at least develop a basic understanding of ideas like routing, IP addressing, TCP and UDP delivery protocols, and DNS management. Plus, understanding the importance of network security with IoT devices and why those things should (or should not) have public-facing IP addresses and internet access goes a long way in helping alleviate concerns from an IT team.
There are many instances where it actually makes more sense to outsource IT management to a third party…
Similarly, it would be wise for church IT and tech support teams to develop an awareness of Production-based network concepts like NDI, ST2110, streaming ACN, ArtNet, and Dante to more clearly understand the needs and requirements of AV teams as they look to function more effectively.
In addition to his role at Granger, Powell leads the Church IT Network, an organization that not only facilitates annual gatherings to help address pertinent topics like these in the church IT and tech spaces, but also maintains a Discord community so peers can collaborate to learn from each other as needed. This idea of overcoming the “you don’t know what you don’t know” is critical to helping ministries stay successful as they grow.
Without having a semblance of empathy to understand what it’s like in another team’s shoes, the overall ministry will ultimately suffer because the teams involved aren’t able to really understand the steps they need to take in order to help the other be successful.
“Too many times there is a complete misunderstanding of the requirements and needs of AV over IT,” says Second Baptist’s Brown. “Not all network traffic is the same. IT understanding AV protocols goes a long way to building trust. Both disciplines need to be working together for the common goal of spreading the Gospel.”
“It’s in the best interest of the Church when these two teams are functioning well together, not just in terms of cost, but in the overall culture of both teams,” emphasizes Glover.
“I think it really comes down to building a culture of working together. Both teams need to develop a ‘what if’ mindset.” —Matt Glover, Director of Technology at Birmingham, Ala.’s Church at Brook Hills.
“The Production team may implement a solution that meets their immediate needs, but did they do it in a way that is secure? Did they do it in a way that won't cause issues on the network? These are questions that can only be answered by working together with IT. A lack of collaboration can lead to duplication of resources, as well as security issues. What if IT already has a corporate Dropbox subscription, or another file-sharing solution in play? If Production sets up their own Dropbox subscription, they’ve now duplicated resources. Segmented solutions are often unmanaged and can lead to security issues as well. What happens when the Production team member that setup the Dropbox account leaves and doesn’t share the password? It’s always in the best interest of the Church to work together to build systems that not only scale, but that can be actively managed in a secure way, regardless of staff changes,” he concludes.
There ultimately isn’t a one-size-fits-all approach to network management for AV and IT needs. For some ministries with the budget, staffing, and resources, it may be feasible to keep the two worlds completely isolated, maintained by different teams, so that protocols and requirements of each don’t step on the other’s toes.
However, many ministries likely aren’t able to make that level of investment, making it even more critical that teams learn how to collaborate to have all devices cohabitating effectively on the same network with VLANs set up to manage traffic flow and firewall policies in place to help keep the network secure.
But in order for collaboration to work, there also has to be documentation in addition to communication. What devices are on the network? What’s the basic outline of an IP scheme? What are the passwords for hardware and software platforms that may need to be accessed or managed?
Like with any other aspect of church technology, managing a network takes time, and that’s arguably the most limited and precious commodity for a team member. There are many instances where it actually makes more sense to outsource IT management to a third party, similar to using an AV integrator to design and install a new system.
“If you don’t have some knowledge of established protocols and what’s on the horizon, you will make poorly informed purchasing decisions and be left behind,” cautions Shawn Kirsch, Tech Director, Ransom Church, Sioux Falls, South Dakota.
“IT Integrators face a lot of problems every day,” says Shawn Kirsch, the Tech Director at South Dakota’s Ransom Church. “They already know how to fix the problem you need to research. Even if you want to manage it yourself, bring someone in to do the initial setup and major upgrades. They can make sure everything is functioning properly so that anything you need to manage is minimal. They will also help to make sure everything is compatible before it’s purchased. Downtime for the church will be minimized and you will save countless hours to focus on your other projects.”
Granger’s Powell adds, “I've used several church focused IT Managed Service Providers (MSPs) over the years for various project work with great success. Then about a year ago my Network Administrator took a new job leaving me with a vacancy to fill. Instead of trying to hire someone, I decided to instead contract [a company] to backfill much of that role. Today [that] team helps us manage, support, and monitor our infrastructure (firewalls, switches, servers, Azure/M365, Multi-Device Management) and some of our user-facing devices.”
“A big part of the job is just knowing what’s possible. You might not be able to afford something, but awareness helps you make decisions that lead you closer to it.” —Shawn Kirsch, Tech Director, Ransom Church, Souix Falls, SD
When limited staff can’t fill in the gaps, there are likely other companies that can. But an effective tech will use that as an opportunity to further educate himself or herself about what is happening in the industry.
“If you don’t have some knowledge of established protocols and what’s on the horizon, you will make poorly informed purchasing decisions and be left behind,” cautions Kirsch. “A big part of the job is just knowing what’s possible. You might not be able to afford something, but awareness helps you make decisions that lead you closer to it.”
For nearly a decade, Brook Hills’ Glover served as the Technical Director, managing just Production aspects before stepping into a broader Director of Technology role in 2014 that also included IT oversight. Having seen both sides of the coin, his perspective speaks to the importance of unity between two typically territorial entities.
“I think it really comes down to building a culture of working together,” he states. “Both teams need to develop a ‘what if’ mindset. What might happen if I make a change, and how might it affect the other team? The Production team should make it a habit to consult with IT when they need to make a change or bring a new piece of networked AV equipment online. The IT team should consult with the Production team when they do something that may affect the operation of Production equipment. It’s probably not best for IT to push a firmware update to a switch on Saturday night or adjust QoS settings without an understanding how it may impact networked AV devices. When each team is looking out for and respecting the needs of the other team, it leaves room for a healthy relationship to grow.”
