Insert BS here A place to discuss anything you want

How (and why) to Ramble on your goat sideways

Thread Tools
 
Search this Thread
 
Old 03-22-2019, 12:25 AM
  #30241  
Senior Member
 
gooflophaze's Avatar
 
Join Date: May 2007
Location: Atlanta
Posts: 997
Total Cats: 156
Default

Eh, take the time and write down all your notes for a post mortem. Always include what went well, what didn't work well, and how this could be avoided in the future (new process or checks) and if there was a recovery plan in place.
gooflophaze is offline  
Old 03-22-2019, 06:53 AM
  #30242  
Senior Member
iTrader: (5)
 
chiefmg's Avatar
 
Join Date: Dec 2010
Posts: 1,474
Total Cats: 1,113
Default

Pro marine engineering tip of the day: Check valves work much better when they are installed in the correct orientation. This means not backwards and not upside down.
chiefmg is offline  
Old 03-22-2019, 08:28 AM
  #30243  
Boost Pope
iTrader: (8)
 
Joe Perez's Avatar
 
Join Date: Sep 2005
Location: Chicago. (The less-murder part.)
Posts: 33,071
Total Cats: 6,623
Default

Originally Posted by chiefmg
Pro marine engineering tip of the day: Check valves work much better when they are installed in the correct orientation. This means not backwards and not upside down.
Yikes. I run into the electrical equivalent of this from time to time. Hopefully no engine damage occurred as a result.
Joe Perez is offline  
Old 03-22-2019, 08:34 AM
  #30244  
Elite Member
iTrader: (5)
 
Erat's Avatar
 
Join Date: Oct 2011
Location: Detroit (the part with no rules or laws)
Posts: 5,677
Total Cats: 800
Default

Gotta love when the FNG puts a 1200a flat pack diode in backwards.
Erat is offline  
Old 03-22-2019, 08:34 AM
  #30245  
Boost Pope
iTrader: (8)
 
Joe Perez's Avatar
 
Join Date: Sep 2005
Location: Chicago. (The less-murder part.)
Posts: 33,071
Total Cats: 6,623
Default

Originally Posted by gooflophaze
Eh, take the time and write down all your notes for a post mortem. Always include what went well, what didn't work well, and how this could be avoided in the future (new process or checks) and if there was a recovery plan in place.
The post-mortem, which I finished around midnight. For reference: The original problem was no dialogue during the 4pm and 5pm episodes of Blue Bloods (but normal music and SFX) when viewed at most (but not all) of the monitoring points in the building, "East" is internal shorthand for the East-coast feed of WGN America, "SAC" means Satellite Air Control (the control room which monitors our uplinks.) "Indy" is our backup site in Indianapolis, which runs an un-manned duplicate of all of our systems, but does not provide Dolby Surround audio (which was an important point.) Also, both the airchain and the monitoring system are hideously (and needlessly) complex, with lots of parallel but unequal paths. Teal and Brown are the designations of one of the paired-redundant master control switcher systems (The others, in case you care, are Blue / Green, Orange / Black, Silver / Gold, Red / Purple, and Stars / Stripes. I have proposed that if we have to install another one, we call it Infrared / Ultraviolet.) "Dirty" is shorthand for a feed which bypasses the main switcher, using a side-router and an external framesync. It has all of the main programming, but lacks transition effects such as squeeze and suckback, as well as the bug and snipes.



Best theory at the moment: the problem is in the East (#1) D9854 in rack 122. Specifically, the downmix to analog out 1/2 of the D9854. (Dwg 644, via dwg 777, input port 5 L/R on the Rk 154 Venus analog audio router, via double-punches at PP 155-A-11 and -12.) This feeds the 3.1 cable modulator (116-4, input 1), the Arbitron monitor in 106 (is this obsolete?), and some, but not all, of the audio monitors in the control rooms. (This may also provide the answer to Question 1 below, for which I shall review the wiring to it tomorrow.)


The Volicon recording of East, which does not show any problems, comes from a different D9824 in Rack 130 (Dwg 946.). It is a direct ASI feed from the downlink via Ant 2, and is a much simpler path than any of the other monitoring, which come from different receivers and have much more complexity.

We did not hear a problem in the primary East audio monitor in SAC (the Genelec / Studio Technologies surround monitor system), because it was listening to a different path. Specifically, a six-channel analog conversion of a four-pair discrete AES/EBU feed, via an Nvision 5128 switcher, of a de-embedded AES/EBU stream of what I assume to be a Dolby Digital (consumer-type, not Dolby E) output of the receiver in rack 122. (Drawings 644, 646, 943, and 946, in that order.)


When we switched from Teal to Brown, the problem appeared to go away briefly, then return. The reason for this is that, in the process of switching between colors, the bypass switcher goes to the Dirty FS immediately, then switches back to the Nvision (on the target color) at the next scheduled break point. The Dirty FS inherently strips off Dolby E and passes only the two-channel mix.


While troubleshooting, I failed to account for the fact that at every point after the NVision, the audio is six-channel discrete (embedded) during Dolby E programming, but switches to two channel during non-surround programming, and our test instruments defaulted to channel 1/2 only. (Thanks to Juds for that insight.)

Indy's incoming signal sounded good to us because it's two-channel stereo at all times.


So a genuine failure did occur in our monitoring system, and the coincidental fact that the (perfectly functioning) normal airchain path seemed to mimic that failure when we listened to it with two-channel test instruments caused confusion. Lesson for the future: the fancy 2RU Wohler on the cart at R200 (East Tape, in front of the NV9000), while perfectly capable of six-channel monitoring, apparently defaults to two-channel for the actual "this is what you can hear through my built-in speakers" part. Ditto the Phabrix.



Things I want to understand:

1: Why does the 1RU Wohler in SAC, which is the primary audio monitor for West, not fail when West is in surround programming on input 1, but does fail when switched to input 2 (East) when in surround?

2: What changed today in the audio feed into the cable modulator? (Ref comments above about the D9854 in rack 122)

3: Why are audio & video separate going into the 3.1 / 3.2 cable modulator? (Dwg 275.)

4: Why do we have so many points where we de-embed / re-embed the audio, in separate paths which we treat as being identical?

Last edited by Joe Perez; 03-22-2019 at 09:34 AM.
Joe Perez is offline  
Old 03-22-2019, 08:37 AM
  #30246  
Boost Pope
iTrader: (8)
 
Joe Perez's Avatar
 
Join Date: Sep 2005
Location: Chicago. (The less-murder part.)
Posts: 33,071
Total Cats: 6,623
Default

Originally Posted by Erat
Gotta love when the FNG puts a 1200a flat pack diode in backwards.
I gotta ask what the application is for a 1200a diode. (Welding? Electroplating, maybe?)

Also, having looked at Digikey, I'm a bit surprised by how inexpensive these are. In my line of work, I'm accustomed to diodes reverse-rated at >30 Kv, but usually at only a few forward amps.
Joe Perez is offline  
Old 03-22-2019, 10:19 AM
  #30247  
Moderator
iTrader: (12)
 
sixshooter's Avatar
 
Join Date: Nov 2008
Location: Tampa, Florida
Posts: 20,675
Total Cats: 3,017
Default

Joe, simplification of the cobbled system should be your legacy. Thirty year old components will be 40 years old before you know it. Additional layers will inevitably be added. Needs a streamlining from what you have said.
sixshooter is offline  
Old 03-22-2019, 10:51 AM
  #30248  
Elite Member
iTrader: (5)
 
Erat's Avatar
 
Join Date: Oct 2011
Location: Detroit (the part with no rules or laws)
Posts: 5,677
Total Cats: 800
Default

Originally Posted by Joe Perez
I gotta ask what the application is for a 1200a diode. (Welding? Electroplating, maybe?)

Also, having looked at Digikey, I'm a bit surprised by how inexpensive these are. In my line of work, I'm accustomed to diodes reverse-rated at >30 Kv, but usually at only a few forward amps.
Yeah. Rectifier for anodizing.

Here's a 7000a 600v diode next to a 1200a 1200v SCR. I gotta be careful because this is about. $1500.


Erat is offline  
Old 03-22-2019, 12:43 PM
  #30249  
Boost Pope
iTrader: (8)
 
Joe Perez's Avatar
 
Join Date: Sep 2005
Location: Chicago. (The less-murder part.)
Posts: 33,071
Total Cats: 6,623
Default

Originally Posted by Erat
a 1200a 1200v SCR

(Does math. Concludes 1.4 MVA. Thinks about the fact that Erat is turning 1.4 MVA on and off with a friggin' semiconductor.)


Joe Perez is offline  
Old 03-22-2019, 12:51 PM
  #30250  
Elite Member
iTrader: (2)
 
Godless Commie's Avatar
 
Join Date: Jun 2009
Location: Istanbul, Turkey
Posts: 3,214
Total Cats: 1,687
Default

Here's an English link that explains what we do on the new show...


World Water Day is on March 22. TRT's documentary division has marked the day with a new series focusing on water finding efforts of a duo travelling the world.


Turkish Radio and Television’s documentary channel, TRT Belgesel, has marked World Water Day with a brand new documentary focusing on the importance of water. Noting that the lives of women and children in particular are spent trying to access clean water sources, the documentary Su Savaslari (literally, Water Wars, translated as Water Walk) is launching to make a difference.

The documentary sets out to change lives in the world, in villages many have not heard of, to unite water with those who need it the most. It provides not only water, but in a way, a future, an education, a brand new life.

The hosts, Caglar Demirkapi and Hakan Girginer, spring to action for change. Demirkapi and Girginer allow us to witness amazing cultures and lives while they are on the path to seek water.

The
in Su Savaslari is Niger, where the team steps into a brand new world with young Fatouma, as they follow her walk to the water source. The hosts have a moment of enlightenment as they realise their privilege and that what they take for granted at home can be considered a gift from God anywhere else.

Turkish NGOs have until today solved the water problems for more than 25,000 villages in Africa alone. This has happened, and will continue to happen, with the support of the Republic of Turkey and charitable Turkish citizens.

Godless Commie is offline  
Old 03-22-2019, 01:17 PM
  #30251  
Elite Member
iTrader: (5)
 
Erat's Avatar
 
Join Date: Oct 2011
Location: Detroit (the part with no rules or laws)
Posts: 5,677
Total Cats: 800
Default

Originally Posted by Joe Perez
(Does math. Concludes 1.4 MVA. Thinks about the fact that Erat is turning 1.4 MVA on and off with a friggin' semiconductor.)

You should see the firing board for them.

Each rectifier has 6, 2 per phase(both sides of the sine wave). So I think you have to divide that number by 6. Or by 3.

Edit*

It's also only controlling around 1200a @ 480v. I'm not sure why they're rated so high. I'm not an electrical engineer.
Erat is offline  
Old 03-22-2019, 02:46 PM
  #30252  
Boost Pope
iTrader: (8)
 
Joe Perez's Avatar
 
Join Date: Sep 2005
Location: Chicago. (The less-murder part.)
Posts: 33,071
Total Cats: 6,623
Default

Originally Posted by Erat
It's also only controlling around 1200a @ 480v.


Last edited by Joe Perez; 03-23-2019 at 02:17 AM.
Joe Perez is offline  
Old 03-22-2019, 04:33 PM
  #30253  
y8s
2 Props,3 Dildos,& 1 Cat
iTrader: (8)
 
y8s's Avatar
 
Join Date: Jun 2005
Location: Fake Virginia
Posts: 19,338
Total Cats: 573
Default

Originally Posted by z31maniac
Side note: This link is garbage, go to his profile sourceforge to download (for anyone interested). That link you shared forces you to accept a bunch of Chrome extensions or it cancels the download.

https://sourceforge.net/projects/equ...atest/download
I didn't see the same thing. But then I block ads 6 ways from sunday.
y8s is offline  
Old 03-22-2019, 05:19 PM
  #30254  
Senior Member
iTrader: (5)
 
chiefmg's Avatar
 
Join Date: Dec 2010
Posts: 1,474
Total Cats: 1,113
Default

Originally Posted by Joe Perez
Yikes. I run into the electrical equivalent of this from time to time. Hopefully no engine damage occurred as a result.
The whole story is pretty convoluted and I don't feel like typing it. Basically some piping got replaced and a contractor installed the check valve incorrectly. No one on board apparently checked the work, for whatever reason. No damage to the engine, when it's running we monitor what's going on which is how we found out the cooling water flow was not what one would expect.
chiefmg is offline  
Old 03-22-2019, 07:01 PM
  #30255  
Boost Pope
iTrader: (8)
 
Joe Perez's Avatar
 
Join Date: Sep 2005
Location: Chicago. (The less-murder part.)
Posts: 33,071
Total Cats: 6,623
Default

EDIT: I apologize for the fact that the formatting of this post makes it look like it was generated by a cat walking across my keyboard. I assure you that it looked fine when I previewed it (paragraph spacing, indentation, etc,) and that what you are about to read is partly a product of the fact that Internet Brands (as a whole) and @Robb M. (specifically) are a bunch of Satan-worshiping serial child-rapists who want nothing more than to rob us all of joy and happiness. (And, presumably, to drive all users away from MiataTurbo and cause the forum to collapse.)



Originally Posted by chiefmg
The whole story is pretty convoluted and I don't feel like typing it. Basically some piping got replaced and a contractor installed the check valve incorrectly. No one on board apparently checked the work, for whatever reason.
Maybe I'm naive, but I guess I've always assumed that in the context of a marine engine large enough to be supervised during operation, repairs would be both supervised and also checked. That's certainly how they do it at the yard during construction. Every single nut & bolt is supervised by both a yard foreman and a manufacturer's representative, and then it's checked by a yard inspector and a different' manufacturer's rep. (Admittedly, my experience here is limited to a single shipyard, Meyer Werft, and two ships, both powered by MAN and Siemens.)


A coda to my post from last night:

The problem turned out to be hilariously complex, in that there was no actual problem. As with most great engineering catastrophes in history, it was a symphony of many failures which all came together in just the right way.



We have our own in-house cable TV system, which services a few hundred TVs throughout the building. It's like a miniature cable company head-end. Four racks of modulators and combiners and amplifiers and splitters, located in the middle of the main equipment room. Hotels and hospitals and cruise ships and prisons have similar setups.

Most of the cable modulators take an HD-SDI or HDMI feed, with embedded two-channel audio, and that's it. A few of the feeds, however, come from sources which originated with Dolby E* multichannel sound, and thus have discrete six / eight-channel audio embedded on the SDI stream. Our in-house standard configuration for multichannel audio in the TRANSPORT phase is:

  • 1a: Left front (or Left, if not surround)
  • 1b: Right front (or Right, if not surround)
  • 2a: Center
  • 2b: LFE
  • 3a: Left rear
  • 3b: Right rear
  • 4a: SAP
  • 4b: Timecode

(Note that all of these are on a single wire, along with the video.)

It's entirely possible to cram all of that into a single QAM cable channel, but some TVs don't decode it properly. You wind up hearing SAP and Timecode mixed in. (For consumer-end purposes, we encode SAP differently, and timecode is never broadcast except as part of the PSIP stream, which is separate from the audio.) So for these sources, we externally downmix pairs 1-3 into a single (discrete, analog) stereo pair, and then feed that into the modulator separately from the video. We then tell the modulator to discard the audio embedded in the SDI stream, and use the discrete analog audio instead. It's a kludge, but it works well.

With me so far?

At the STORAGE phase, all of the source material which contains surround is encoded with regular stereo (Lt / Rt) on AES channel 1, and the Dolby E surround track on AES channel 3. So, the tapes / discs / files containing all of the programming are formatted as:






  • 1a: Left (total)
  • 1b: Right (total)
  • 2a: silence
  • 2b: silence
  • 3a & 3b: combined Dolby E signal
  • 4a: SAP
  • 4b: Timecode

The master control switchers are set up to detect whether a valid Dolby E track is present on input channel 3. If it is, then they decode it and spit it out as discrete six-channel audio on output channels 1-3 as noted previously in the transport phase. If it isn't, then they just pass input channel 1 to output channel 1, and leave channels 2 & 3 silent. If the Dolby E decoder fails, they revert back to Lt/Rt.

The MPEG encoders which feed the satellite uplink are configured to recognize which format they are receiving, and adjust accordingly. If they are getting six channel audio, then they produce both a stereo (Lt / Rt) mix as well as a Dolby AC3 stream, and squirt both of those up to the bird. (Your TV / home theater system does magic to select the right one and decode it.) If they are getting two-channel audio, then they just produce a straight two-channel feed, and turn off the bit in the datastream that tells your TV to look for an AC3 stream.

WGN America is the third generation of satellite program to originate from this location. It started out in 1978 as "The Superstation," when it was just a simulcast of "WGN, Chicago's Very Own." Obviously, there have been a lot of changes over the past forty years. Over time, equipment has been added and removed and added and removed and added and removed, and we've wound up with what may be the most horrifyingly complex signal path in the history of broadcasting. A well-designed airchain should be able to be sketched on a letter-sized sheet of paper, with a pencil. WGNA's signal path occupies NINE E-sized (34 × 44 inches) sheets of plotter paper.

Needless to say, it's an amazingly complex machine.


Around 4pm, the operations manager of WGN America comes into my office saying that there's music and SFX, but no dialogue, from the point of view of the TV in his office. I turn up the volume on one of the five TVs in my office, and get the same result. I go into master, and hear dialogue, but I know from experience that their monitor path is slightly separate from the main feed.

I run into the machine room, grab a piece of test equipment, and plug it into a monitor point in the airchain. I hear no dialogue. I plug it into a monitor point on the server that's playing the episode. I hear dialogue. So I switch from the primary to the secondary airchain. No change.

I then switch from the Teal switcher to the Brown switcher. Miraculously, everything is fixed.

Until the end of the next commercial break. Then it's gone again. I continue patching through the airchain, using five separate pieces of test gear, and all give me the same result. The bargraph meters show activity on all six channels, but according to ear, no dialogue.



Lots and lots of **** happened, and then 6pm came around, and it was Married with Children, and no surround for the rest of the night. So we stopped to reflect.




Here's what I finally determined:


FAILURE 1:

The video feed which we plumb into the house cable system is a decoded version of the uplink stream. And, since it contains the eight-channel audio (which doesn't agree with some TVs), we externally crunch the first six channels down into a stereo pair, and feed that into the modulator separately.

One of my engineers has been assigned, for several months, to cleaning up the house cable system. We recently removed the last of the analog TVs that were connected to it, so we're in the process of ripping out all of the old analog modulators and their associated wiring, revising the documentation to reflect this, and consolidating the digital modulators to free up some rack space.
Yesterday, at around 9am, this engineer noticed that the 116-4 modulator was set to external audio, and so he CHANGED IT TO EMBEDDED AUDIO, SO THAT IT WOULD BE LIKE THE ONES ABOVE AND BELOW IT. AND HE DID NOT TELL ME ABOUT THIS, OR MAKE A NOTE IN THE SHIFT LOG.
Nothing bad happened immediately, since at that time of day, WGNA is airing reruns like Murder She Wrote, In The Heat Of The Night, and the like. None of it has surround



FAILURE 2:

Every single one of the five pieces of teat gear I grabbed had only been configured to output the audio of channel 1a/1b. During non-surround programming, that's fine. Left and Right. But during surround programming, that's only Lf / Rf, and excludes the center channel, which is where the dialogue normally is.



FAILURE 3:

I learned something new about our master control switchers. When you command it to switch from one color to the other, it doesn't just do it all in a single step. First, it switches to a sidechain feed of the target color, bypassing the actual switcher itself through an external framesync. Then it reconfigures the switcher for the new color, and then, at the next breakpoint in the log (eg: a commercial) it switches off the framesync and back to the main switcher. And guess what: the framesyncs are not sophisticated enough to decode Dolby E, so they just throw it away and pass Lt / Rt straight through on audio channel 1. That's why switching from Teal to Brown magically "fixed" the nonexistant problem, only to "break" again after the next commercial block.





I am exhausted as hell, I didn't need this **** right now (we're prepping to host a debate between the two remaining candidates for Mayor on Monday, and since the Mayor of Chicago is basically an elected Mafia boss, it's gonna be interesting), and I am now officially going to start drinking.




Also, a rectifier stack:



When dealing with tens of kilovolts and just a few amps, they arrange many diodes in series. This is a three-phase rectifier from an FM transmitter.






















* = The Wikipedia article states that "Dolby E never reaches home viewers as it is intended for use during post-production when moving multichannel material between production facilities or broadcasters." It should read "... should never reach home viewers..." There have been numerous cases of people such as myself doing something utterly retarded which results in the stream going out on the air as the Ls / Rs channel. And yes, we have had to pay for the replacement of many viewer's speakers, as well as a few tort claims for personal injury, as a result.
Joe Perez is offline  
Old 03-22-2019, 08:52 PM
  #30256  
Senior Member
 
gooflophaze's Avatar
 
Join Date: May 2007
Location: Atlanta
Posts: 997
Total Cats: 156
Default

Yeap - that's about my experience with post mortems. It's usually no single fault but an error stack that combine together.

And now you know, and knowing is half the battle. Even if it does increase the urge to damage your liver - cuz hey, that's why you're paid the big bucks.

*Addendum: Legacy systems bite you in the *** every damn time.
gooflophaze is offline  
Old 03-22-2019, 10:40 PM
  #30257  
Elite Member
iTrader: (21)
 
rleete's Avatar
 
Join Date: Jun 2007
Location: Rochester, NY
Posts: 6,602
Total Cats: 1,264
Default

All I can say is, it's about damned time you earned your salary.
rleete is offline  
Old 03-23-2019, 03:19 PM
  #30258  
Senior Member
iTrader: (5)
 
chiefmg's Avatar
 
Join Date: Dec 2010
Posts: 1,474
Total Cats: 1,113
Default

Originally Posted by Joe Perez
Maybe I'm naive, but I guess I've always assumed that in the context of a marine engine large enough to be supervised during operation, repairs would be both supervised and also checked. That's certainly how they do it at the yard during construction. Every single nut & bolt is supervised by both a yard foreman and a manufacturer's representative, and then it's checked by a yard inspector and a different' manufacturer's rep. (Admittedly, my experience here is limited to a single shipyard, Meyer Werft, and two ships, both powered by MAN and Siemens.)
You aren't naive. That is normally the way things work. However, this ship is owned by the US government and is in a laid-up state with minimal outlay of cash for parts and work to be accomplished. Under those circumstances all bets are off. There have been so many people working here (as crew) who don't care, and the pay is low so the jobs aren't in demand. That adds up to a recipe for a lot of ****-ups. In this case some pipework was done and no one double-checked on the final product. There have also been cases on this ship of water being put into a ballast tank that had a manhole access left loose from when it had last been inspected, that resulted in a wee bit of flooding...More than once (different tanks). I am very glad I only agreed to do four months on here. On the plus side we are making progress towards having things the way they should be.


A whole bunch of electronic/communication stuff I got a fleeting grasp of: On the bright side, you know now something more than you did about how it happened and what to do should it happen again. That puts you a leg up.
chiefmg is offline  
Old 03-26-2019, 02:50 PM
  #30259  
Boost Pope
iTrader: (8)
 
Joe Perez's Avatar
 
Join Date: Sep 2005
Location: Chicago. (The less-murder part.)
Posts: 33,071
Total Cats: 6,623
Default

Can anyone here offer practical advice on how to poison a squirrel? Or, more specifically, a lot of squirrels?
Joe Perez is offline  
Old 03-26-2019, 03:10 PM
  #30260  
Junior Member
iTrader: (1)
 
Guardiola's Avatar
 
Join Date: Jan 2014
Location: Raleigh, NC
Posts: 286
Total Cats: 31
Default

Originally Posted by Joe Perez
Can anyone here offer practical advice on how to poison a squirrel? Or, more specifically, a lot of squirrels?
Trapping is your only option. I would tell you to go to your local farm supply store, but being in Chicago, Amazon might be your best bet.
Peanut-butter works well for bait.

Guardiola is offline  


Quick Reply: How (and why) to Ramble on your goat sideways



All times are GMT -4. The time now is 09:26 PM.