,

Tip #1300: A Hidden SSD Speed Boost

… for Codecs & Media

Tip #1300: A Hidden SSD Speed Boost

Larry Jordan – LarryJordan.com

SSDs don’t have seek times or latency. This means MUCH faster storage speeds for multiple apps accessing storage at once.

Samsung T-5 SSD speed in isolation (top) and with BMD and AJA both running (bottom).

Topic $TipTopic

OK, I admit, I was playing. But I discovered something very intriguing about SSDs. Watch.

As we’ve learned over the last several tips, the speed of spinning hard drives are limited by seek times and latency (Tip #1287)

The speed of a network is limited by how the devices are connected to it (i.e. 100 Mb vs. 1 Gb vs. 10 Gb Ethernet), the number of users and the connected speed of the server.

But, direct-connected SSDs don’t have these limitations. Instead, speeds are controlled based upon the construction and connection protocol of the SSD (PCIe vs. NVMe – and – USB vs. Thunderbolt).

I plan to do this test in more detail in a few weeks, when I get a chance to play with a brand-new, high-performance NVMe SSD.

But, for this quick check, I connected a Samsung T-5 SSD to a 2019 Mac mini running Thunderbolt 3. While the Thunderbolt 3 protocol maxes out around 2500 MB/sec, the T-5 pegged the meter at 479 MB/s write and 526 MB/s read (see the top values in the screen shot).

HOWEVER, when I ran BOTH AJA System Test and Blackmagic Disk Speed Test at the same time, while the speed for each application dropped, the aggregate speed was FASTER than the speed for the isolated test.

NOTE: In my example, the single app read speed was about 525 MB/s. When both apps were running, the aggregate speed was about 660 MB/s!

What this means is that if you have multiple applications reading or writing to SSD storage at the same time – which is typical for many media apps – SSDs provide far less of a slow-down than spinning media because we can access all that storage directly, without waiting for platters to spin and heads to jump into place.

These tests are just preliminary – I’ll have more on this in a few weeks. But I think this is very, VERY interesting!

Please rate the helpfulness of this tip.

Click on a star to rate it!

/2 Comments/by
,

Tip #1286: Optimize RAIDs for SSDs

… for Codecs & Media

Tip #1286: Optimize RAIDs for SSDs

Larry Jordan – LarryJordan.com

Optimizing RAIDs for SSD drives will improve performance.

(Image courtesy of OWC.)

Topic $TipTopic

RAIDs (Redundant Array of Independent Disks/Drives/Devices) consist of a number of hard drives or SSDs grouped together into a single unit so that they appear to the computer as a single device. Because there is more than one hard drive in a RAID, they offer greater performance and storage.

As we continue shifting storage from spinning media, which holds a LOT, to SSDs, which are FAR faster and more flexible, we need to rethink how RAIDs are configured. This configuration is done using “levels.” There are different levels of RAIDs, identified by numbers:

NOTE: Another benefit to SSD-based RAIDs, is that there is no latency. Because there is no mechanical movement, data can be retrieved much more quickly.

  • RAID 0 – Fast, inexpensive, no data redundancy. Requires a minimum of two hard drives inside the RAID enclosure. The more drives you add, the faster the performance, as performance and storage capacity are the sum of all drives in the RAID. However, if you lose one drive, you’ve lost ALL your data. Most often used when speed combined with low cost are paramount.
  • RAID 1 – Complete data redundancy. Generally only uses two hard drives inside the RAID enclosure. Often called “mirroring,” each drive is a complete copy of the other. Most often used for backing up servers or when on-set for DIT media work. Has the speed and capacity of the slowest single drive in the system.
  • RAID 3 – Medium-fast, data redundancy. Requires a minimum of three drives, as one drive is reserved solely for parity data. Should one drive die, your data is safe. This technology is no longer in common use, replaced by the faster performance of RAID 4 or 5 systems.
  • RAID 4 – Very-fast, data redundancy. Similar to RAID 3, requires a minimum of three drives, as one drive is reserved solely for parity data. Should one drive die, your data is safe. This is the preferred RAID format for SSD drives because of how the data is stored on the drives. When compared to a RAID 5, RAID 4 with SSDs is about 25% faster on reads.
  • RAID 5 – Very fast, data redundancy. Requires a minimum of three drives and shares parity data across all drives. Most often found with four or more drives inside. If one drive goes down, your data is safe. This is the preferred choice for RAIDs containing spinning media (traditional hard disks). Used for both locally-attached storage and servers.
  • RAID 6 – Fast, extra data redundancy. Requires a minimum of four drives. This version protects your data in the event two hard drives die at the same time. More expensive than RAID 5, but, generally, the same physical size. Like the RAID 5 this is most often used connected to just one computer. Not as fast as a RAID 5.
  • RAID 10 (or 1+0) – VERY fast, totally redundant. Requires a minimum of four drives, but is more often created by combining two matched RAID 0’s into a RAID 1. This provides the speed equivalent of a RAID 0, with the data redundancy of RAID 1. As RAIDs continue to drop in price, this can be a less-expensive way to create systems that rival the performance of a RAID 50.
  • RAID 50 – VERY fast, data redundancy. Generally the domain of very large RAIDs, this format combines the speed of RAID 0 with the redundancy of RAID 5 by dividing the RAID into sections, where you can lose a drive in each section without losing data. These systems generally cost more than $10,000 and contain at least twelve drives. Generally used in network and server situations where multiple users need to access the same data.
  • RAID 60 – VERY fast, extra data redundancy. Generally the domain of very large RAIDs, this format combines the speed of RAID 0 with the redundancy of RAID 6 by dividing the RAID into sections, where you can lose two drives in each section without losing data. These systems generally cost more than $10,000 and contain at least twelve drives. Generally used in network and server situations where multiple users need to access the same data.

EXTRA CREDIT

RAID 4 is the preferred option for SSD-based RAIDs.

/0 Comments/by
,

Tip #1287: What is Latency

… for Codecs & Media

Tip #1287: What is “Latency?”

Larry Jordan – LarryJordan.com

Hard drives have latency – SSDs do not.

(Image courtesy of Pexels.com.)

Topic $TipTopic

Latency is directly tied to spinning storage media – the traditional hard drive – and determines how quickly you can access your data. Latency is the average time for the data being accessed to rotate into position under the drive’s magnetic head, after a completed seek.

As PC Tech Guide.com writes:

Disk RPM is a critical component of hard drive performance because it directly impacts the latency and the disk transfer rate. The faster the disk spins, the more data passes under the magnetic heads that read the data; the slower the RPM, the higher the mechanical latencies. Hard drives only spin at one constant speed, and for some time most fast EIDE hard disks spin at 5,400 rpm, while a fast SCSI drive is capable of 7,200 rpm.

Mechanical latencies, measured in milliseconds, include both seek time and rotational latency. Seek Time defines the amount of time it takes a hard drive’s read/write head to find the physical location of a piece of data on the disk. Latency is the average time for the sector being accessed to rotate into position under a head, after a completed seek. It is easily calculated from the spindle speed, being the time for half a rotation.

A drive’s average access time is the interval between the time a request for data is made by the system and the time the data is available from the drive. Access time includes the actual seek time, rotational latency, and command processing overhead time.

EXTRA CREDIT

What makes SSDs so fast is that they don’t spin or have magnetic drive heads. This means that terms like latency and seek time no longer apply. Here’s the full PCTechGuide.com article to learn more.

/0 Comments/by
,

Tip #1267: Top Filmmaking Gear for 2020

… for Random Weirdness

Tip #1267: Top Filmmaking Gear for 2020

Larry Jordan – LarryJordan.com

Every list is subjective, share your favorites in the comments.

The Sony A7S III, without a lens.

Topic $TipTopic

This article, written by Lewis McGregor, first appeared in PremiumBeat.com. This is a summary.

Like most industries, the video gear market was inundated with new gear this year. As a filmmaker who often works either as a lone operator or in a skeleton crew, I’m looking for equipment that condenses tasks and increases my efficiency. The less I carry, the better. The gear I’ve highlighted in this list echoes that ethos. Ranked in no particular order, these include:

  • Sony A7S III – $3,498. The next iteration of the filmmaking variant of the Sony A7 line.
  • DJI RS2 – $849. The DJI RS 2 is the successor to the Ronin-S. It’s lighter by design, weighing just 2.36 pounds, but it can carry up to a ten-pound rig.
  • DaVinci Resolve Speed Editor – $295. Like the 2019 DaVinci Resolve Keyboard, the Speed Editor is a peripheral that gives the editor precise and efficient control of the timeline with dedicated function keys and a multi-operational search dial.
  • NVIDIA RTX 3000 series – $499/$699/$1,499. While it may seem initially perplexing to include a line of new GPUs for an end-of-year filmmaking equipment list, you have to acknowledge that with the increase in camera resolution and RAW recording, 2015 GPUs and CPUs aren’t cutting it anymore.
  • Aputure 600D – $1,890. A single chip LED fixture with a reflector. Like the 120D and 300D, the 600D also packs a punch, but it hits a lot harder.
  • Nova P300C – $1,699. What makes this light specifically unique and highly anticipated is that it’s an RGBWW light. That means it can just about integrate into any ambient light situation, match other light fixtures, or just for creative expression, switch to the millions of colors found with the RGB wheel.
  • Canon EOS C70 – $5,499. This camera is something of a hybrid of the new EOS R line and their cine camera line.
  • Blackmagic URSA Mini Pro 12K – $9,995. In 2020, we were treated to the first 12K camera, with the URSA Mini Pro 12K. The body largely remains the same design as the other readily available models. However, the internal electronics of the 12K have been replaced. There’s a new sensor, a new film curve, new color science, and a whole new host of recording features.
  • Fuji XF 50mm F/1.0 – $1,499. The Fuji F/1.0 is an unprecedented entry from Fuji as it marks the arrival of the fastest autofocus lens for mirrorless cameras.

EXTRA CREDIT

The article, linked at the top, has videos demoing all this gear, as well as more specs and details.

/0 Comments/by
,

Tip #1255: Criteria for Buying a Computer System

… for Codecs & Media

Tip #1255: Criteria for Buying a Computer System

Larry Jordan – LarryJordan.com

The challenge is not the final export, but in assembling the pieces to create it.

Topic $TipTopic

Just a couple of minutes ago, I got this email:

“I need a [newer] system to work with now [while waiting for the new Apple silicon systems to be released]. What are your thoughts about using an external SSD with a 2019 mid level iMac 27?”

So, I sent this response:

“Smile…. Until you give me a clue about what you want to do with this gear, it’s pretty darn hard to offer an opinion.”

They then responded:

“Majority of work is for YouTube.”

I replied:

“Key criteria for any hardware purchase are: the speed you need to get things done, the NLE you are using, the frame size you are working in, and the codecs you are using. The distribution format is trivial.

“If you are at 4K and below, not emphasizing HDR and have reasonable deadlines, the 2019 27″ iMac is an excellent choice.”

I mention this conversation because it is a question that I get almost every day – and it’s the wrong question. When buying new gear, we need to have a reasonable idea of what we are using it for. In almost all cases, the end result is not where the work is – it’s in assembling and combining all the pieces.

A sports car, pickup truck and school bus are all potentially excellent vehicles, but only one will do a good job transporting 40 people from Point A to Point B.

/0 Comments/by
,

Tip #1256: What’s the “Ideal” Computer?

… for Codecs & Media

Tip #1256: What’s the “Ideal” Computer?

Larry Jordan – LarryJordan.com

It doesn’t have to be perfect, it simply needs to get the job done efficiently.

(Image courtesy of Pexels.com.)

Topic $TipTopic

My wife has a saying: “The perfect is the enemy of the good.” Nowhere is that more true than in technology; and it is driving us all nuts.

What this saying means is that we spend too much time looking for the perfect system, when a system that may be less than perfect is still more than adequate is enough. This is especially true when it comes to storage.

As an example, I’m in the process of upgrading my server for faster performance and greater capacity. However, last night, as I was exporting my weekly webinar, I measured how fast Final Cut creates a ProRes 4444 file: 85 MB/second. Even if I had storage that clocked in at NVMe speeds – 2500 MB/sec – my exports would not be any faster, because FCP X can only calculate these files so fast.

1080p media needs less than 40 MB/second to edit, while 4K media needs less than 70 MB/sec. Storage that goes 300 MB/second will edit at the same speed as storage that goes 2500 MB/second.

I’m not saying faster storage is a bad idea, clearly, multicam editing, HDR or larger frame rates require more horsepower than simple HD. However, what I am saying is that we need to ask ourselves a bigger question: Where will extra speed actually help? For example, if I only edit one project a week, spending a lot of money improving export speed is not meaningful compared to the time it takes to edit the project in the first place. Sadly, faster storage does not help me think any faster. I wish it did.

Another example was provided by Gloria. She owns a high-end 2019 Mac Pro. She’s worried that Thunderbolt 4, which hasn’t shipped yet, will make her system obsolete.

Well, ALL computers become obsolete at some point, but when it comes to performance, Thunderbolt 4 is the same as Thunderbolt 3. And, even when new gear is released, as it always is, all our current gear will still work exactly the same as it does now.

I get dozens of emails each week from editors happily editing on Mac Pro systems that are 10-12 years old. Clearly not state of the art, but fully capable of doing the work they need to get done – on time and on budget. I get even more emails from editors stressing over whether they need a 3.2 GHz or 3.3 GHz CPU.

My advice is stop trying for perfection – unless the search itself is something you enjoy. Instead find a system that meets your needs. Most of the time, good enough is also fast enough. And “future-proofing” is a fool’s errand.

Jan Frederickson, of WLS-TV, had a sign on her wall that I think about daily: “It’s better than perfect, it’s done.”

That is a reassuring statement.

/3 Comments/by
,

Tip #1185: What Does LiDAR in an iPhone 12 Do?

… for Visual Effects

Tip #1185: What Does LiDAR in an iPhone 12 Do?

Larry Jordan – LarryJordan.com

LiDAR is a key technology that makes AR believable.

(Image courtesy of Halide.com.)
iPhone LiDAR resolution may be better for mapping rooms, than portraits.

Topic $TipTopic

One of the key new features in the iPhone 12 Pro is LiDAR. Lidar stands for light detection and ranging, and has been around since the 1960’s. It uses lasers to ping off objects and return to the source of the laser, measuring distance by timing the travel, or flight, of the light pulse.

An iPhone sends waves of light pulses out in a spray of infrared dots and can measure each one with its sensor, creating a field of points that map out distances and can “mesh” the dimensions of a space and the objects in it. The light pulses are invisible to the human eye, but you could see them with a night vision camera. It works up to a range of 15 feet (5 meters).

The primary purpose of LiDAR in the iPhone is to improve augmented reality (AR) implementation. It will give apps more useful and accurate information about their surroundings, for smoother, more reliable AR. Even today, there is still a lot this technology can do, not just for augmented reality but games and shopping.

LiDAR actually has many uses across many industries. Archaeologists use it to prepare dig sites and autonomous vehicles rely on it to construct real-time 3D maps of their surroundings. LiDAR has even been used to create highly realistic and accurate maps of race tracks in video games, like Project CARS. Police speed guns also use LiDAR.

There’s an excellent article at halide.com, the developers of Halide, an iPhone camera app, that goes into much more detail showing what LiDAR can do and how it relates to AR and mapping the real world into your camera.

As the Halide authors conclude: “Photography isn’t traditionally taking photos anymore; it’s combining all the data and smarts in our devices into allowing totally new interpretations of what ‘photography’ can mean. And if you’re not excited about that, we’re at a loss!”

EXTRA CREDIT

Here are the references I used for this article:

/0 Comments/by
,

Tip #1132: Not All Thunderbolt Cables are High Speed

… for Codecs & Media

Tip #1132: Not All Thunderbolt Cables are High Speed

Larry Jordan – LarryJordan.com

Thunderbolt 3 cables are not the same – and many slow your data down.

(Image courtesy of Intel.)

Topic $TipTopic

One of the most interesting things I learned recently is that not all Thunderbolt cables deliver the same performance. Even Apple’s cables have significant limitations.

I discovered this in an interview with Larry O’Connor, CEO of OWC, a company that specializes in Thunderbolt peripherals.

Here is an excerpt from our interview:

Currently, USB-C cables don’t support Thunderbolt and most don’t even support full USB 10 Gb/s data speeds. For example, the cable that Apple includes with its laptops only carries USB data at USB-2 480 Mb/s speed, not even USB-3 5 Gb/s!

Also, today, Thunderbolt 3 passive cables only provide 40 Gb/s at lengths up to 0.8 M and 20 Gb/s at lengths longer than 1 meter. (I should note that all passive Thunderbolt cables do support full USB-3.2 10 Gb/s speeds)

Sigh… It gets worse. Thunderbolt 3 cables 1 meter or longer are active, which means they have electronics in them to provide the full 40 Gb/s speed. But, while delivering Thunderbolt 40 Gb/s they only deliver USB-2 480 Mb/s speed (that’s a max of 60 MB/s of total data throughput vs. about 600 MB/s with USB-3 5 Gb/s or about 1,200 MB/s with USB-3.2 10 Gb/s)

So, if you are not getting the speed you expect from a Thunderbolt device, or a device connected using a Thunderbolt cable, the first place to look is the cable itself.

Read the full interview here – it is well worth your time.

/2 Comments/by
,

Tip #1122: HDD RPMs Really Do Make a Difference

… for Codecs & Media

Tip #1122: RPMs Really Do Make a Difference

Larry Jordan – LarryJordan.com

Faster hard drive RPM speeds equals faster data transfer rates.

7200 RPM drives are, on average, 30% faster than 5400 RPM drives.

Topic $TipTopic

I’ve spent the last two weeks optimizing my network and two of my locally-attached RAIDs. In this process, I found myself with two identical fully-optimized 4-drive RAIDs; except one was filled with 7200 RPM drives and the other with 5400 RPM drives.

So, I decided to test them to see how much RPM speed affects data transfer speeds.

What I learned is that 7200 RPM drives increase data transfer speeds by 30% on average. (See screen shot.)

For the complete details on my tests and the results, read this article.

/0 Comments/by
,

Tip #1081: Improve the Responsiveness of a Server

… for Codecs & Media

Tip #1081: Improve the Responsiveness of a Server

Larry Jordan – LarryJordan.com

An NVMe SSD cache card can speed file directory operations and make a server more responsive.

The Synology SNV3500-400G NVMe SSD.

Topic $TipTopic

I’ve spent a lot of time this week thinking about how to improve the speed and responsiveness of my storage. Here’s a relatively inexpensive way to improve the responsiveness of a server, especially for smaller workgroups. When it comes to storage there are three elements we can adjust:

  • Storage capacity – measured in gigabytes – is how much the device holds
  • Bandwidth – the speed it transfers data to and from the computer
  • Responsiveness – how fast it responds to requests for data

Looking at these in more detail:

  • Capacity – measured in terabytes – we are all familiar with. From experience, we know that we can’t have too much capacity. It seems that hard disks are either empty or full.
  • Bandwidth – measured in MB/second – is how fast we can move data from one device to another.
  • Responsiveness – measured in milliseconds – is how fast a storage device responds to a request for data.

Most older people celebrate a birthday with a nice dinner. Me? I upgraded my server. I have a Synology DS 1517+ that’s around 3 years old. I connect to it using 1 Gb Ethernet from a variety of different computers.

One of the problems with my storage is that, because my network only consists of 2-3 users, the server is not heavily used. Which means that a lot of the time the drives stop spinning, or spin slower, to save energy because there’s nothing happening across the network.

This means that when I access a server volume from my Mac, it takes several seconds for it to wake up and display a file directory or open a file.

Most of the time, while annoying, this isn’t a big problem. But, all my media is stored on the server, when I’m doing a live webinar this delay drives me nuts.

So, I finally decided to do something about it: I added an SSD card as a cache to the server. SSDs are marketed to database users as a way to improve the responsiveness of I/O operations. And I’m sure it does that. But there are also benefits to media creators in terms of making the server feel much more responsive.

NOTE: The specific hardware that I added were a Synology E10M20-1 Ethernet Adapter, which also holds a Synology SNV3500-400G 400 GB NVMe SSD to accelerate the storage cache. (The card uses an m.2 form factor.)

I was amazed at the difference. Folders pop open almost instantly, even though the disks are not yet up to speed. Navigating is almost as fast as the internal SSD on my computer. While I haven’t, yet, connected the 10 Gbps Ethernet port – though that’s coming – just adding SSD makes a big difference. I don’t feel like I’m waiting on my storage anymore.

NOTE: The SSD does not speed file transfers, but it does make moving around and finding things inside the server much faster.

If you feel that your server is a bit “laggy,” look into adding an SSD card as a cache. It will make your system feel much peppier, even if your server, like mine, still uses spinning disks for storage.

/0 Comments/by