Scaling #AI Apps

Introduction

0:00 how to scale your AI apps I'm here with

0:03 braul L chandani you're the vice

0:05 president product for AI data center

0:08 Solutions at Juniper and we're here at

0:10 Mobile World Congress it's it's a

0:12 crowded place we're still

0:14 early can you explain a little bit more

Networking challenges in scaling AI

0:16 what are the key challenges from a

0:18 networking perspective if companies

0:20 start scaling their AI

0:22 applications yeah on great question so

0:25 first of all anybody launching an AI

0:27 service has many many different

0:28 challenges but from an INF structure

0:30 perspective what we have seen is that

0:33 organizations are trying to walk away

0:35 from closed ecosystem architectures that

0:37 a single vendor only into more best of

0:41 read Solutions and as they move into

0:43 these best of read solutions they don't

0:44 need the comfort that these Solutions

0:46 are going to work together and even over

0:49 the period of just last one year we've

0:51 seen a lot of democratization of AI

0:53 infrastructure happening and this is

0:54 really happening on two fronts the first

0:56 is on the gpus themselves you know while

0:59 Nvidia is still the clear leader what we

1:01 have seen as Juniper is that over the

1:03 last one year more and more

1:05 organizations are starting to experiment

1:07 on even productize other gpus and

Democratization of AI infrastructure

1:10 accelerators out there whether this is

1:11 AMD or Intel or cerebrus or Sova all the

1:14 other ones out there and the same thing

1:16 is happening on the networking side as

1:18 well you know while most discussions in

1:20 2023 were about I can only deploy infin

1:23 band now we are seeing discussions

1:25 change from why ethernet to how ethernet

1:29 customers more more customers believe

1:30 that ethernet is the preferred choice

1:32 and now just they need to know how they

1:34 want to do it and Juniper is really you

1:36 know taking the lead in evolving

1:38 ethernet along that Journey we were the

1:40 first vendor in the industry to launch

1:42 an oem based 800 gig switch in the

1:45 market and we've been very very

1:46 successful with that and we're also

1:48 helping our customers scale their AI

1:50 data centers with delivering software

1:53 features that deliver a self-optimizing

1:55 ethernet think of it like a Google Maps

1:57 for the network when you see a con you

2:00 avoid going down that path and you go

2:02 down a different path to the network

2:04 it's a nice way to to describe it um we

2:06 see a trend as well GPU as a service um

2:10 AI as a service can you explain a little

2:12 bit more how this works and especially

2:14 from a network perspective yeah Ron uh

2:17 and what we saw over the last one year

2:19 is that there's a new breed of companies

2:22 like there are Cloud companies that are

2:24 coming up and popping up that are

2:25 offering GPU as a service or AI as a

2:27 service now these could be service

2:29 providers who have been you know long

Transition from InfiniBand to Ethernet

2:30 entrenched in their in the market and

2:32 they're trying to go into new markets

2:34 but sometimes these are startups that

2:36 are referred to as neoc Cloud providers

2:38 that are entering that space as well and

2:40 they're really trying to cater to this

2:42 Enterprise need uh while Enterprises are

2:45 rushing to build AI Services Enterprises

2:48 don't have the skill sets themselves to

2:50 do it and in sometimes they don't

2:51 economics or pay grow economics works a

2:54 lot better right so that's why this GPU

2:56 as a service providers are you know

2:57 really taking a lot of shape and in

3:00 order to enable these Juniper actually

3:02 uh over the last week has launched uh an

3:06 offering to enable GPU as a service

3:08 providers and we're really catering to

3:11 three big different needs or three basic

3:13 problems that these providers are facing

3:15 right the first is time to Market many

3:18 of these are you know startups many of

Juniper’s 800G switches & AI data center solutions

3:20 them are you know need to rush to

3:22 monetize their services you know strike

3:23 while the you know Iron is hot uh and

3:26 they're in that rush and what Juniper

3:28 can do is help them bring up their

3:30 services in record time and we're using

3:32 Juniper abra's validated blueprints in

3:35 order to U instantiate an entire cluster

3:38 AI cluster in a few minutes uh and we

3:41 also have integration with open shift so

3:43 anybody any of these GPS or service

3:44 providers who are building uh their

3:46 clusters using kubernetes or open shift

3:49 now can deliver the network as a service

3:52 so that you don't have to kind of

3:53 whenever you turn up a tenant on a GPU

3:55 you don't need to have manually go and

3:56 Stitch the network together it can be

3:58 done as a service right now the second

4:00 problem that these uh uh any AI as a

4:04 service provider will face is that your

4:06 gpus are expensive you know you don't

4:09 want to be uh you know running those at

4:12 lower utilization and any network

4:14 congestion can essentially you know

GPU as a Service & AI as a Service trends

4:16 bring the GPU utilization down and

4:18 you're wasting valuable resources right

4:20 so performance is a key key metric that

4:23 is important for GPU service providers

4:25 and again we have provided the products

4:27 like the 800 gig switches and routing

4:29 products to any enable highspeed

4:31 communication between gpus but we also

4:33 have again in abstra the capability to

4:36 pinpoint with great accuracy congestion

4:38 hotpots and to take remediation action

4:41 against to avoid those congestion

4:43 hotpots in the infrastructure and then

4:46 finally multi-tenant networks is is

4:48 inherent in GPU a service providers they

4:50 are offering their service to multiple

4:52 tenants so security and isolation in the

4:54 net infrastructure is again very very

4:56 important and Juniper has historically

4:59 had had has had a very strong security

5:01 portfolio but we what we have recently

5:03 introduced is a is a SRX 4700 which in

5:06 one Ru form factor fax in 1.6 terabits

5:09 per second of uh firewall performance

5:13 that's unprecedented raw and that's

5:14 unprecedented in the industry so far

5:16 right and we do recommend to our

5:17 customers that applo deploy a defense in

5:20 depth strategy so that even if a host is

5:22 infected you PR prevent lateral

5:24 movements of that malare across multiple

5:27 tenants in your infrastructure by

5:28 deploying evpn X Land Based Fabrics that

5:31 completely isolate tenants in the

5:32 backend fabric so our offering for GPS

5:35 service providers is really around uh

5:37 speed it's about security and it's about

5:39 efficiency yeah and in the beginning you

5:40 were mentioning already the open

5:42 ecosystem what what's the importance of

5:43 your open ecosystem in the whole

5:47 perspective I think I think open is

5:50 going to be very very important I I'll

5:51 give you an example one of our gpos are

5:53 service providers is vulture uh and it's

5:56 a public reference so vulture has

5:58 deployed uh an open ecosystem based gpus

Speed, performance, and security for GPUaaS

6:02 service offering based on amd's Mi 300

6:04 gpus based on broadcom's 400 gig power

6:07 to Nick and 800 gig networking from

6:09 juniper the reason open is important is

6:12 because it helps uh the vendor to the

6:15 the the whoever is building the AI

6:17 cluster to pick best of breed components

6:19 but at low much lower economical cost if

6:22 you're locked into a single vendor the

6:23 costs are just generally going to go

6:25 higher right and Juniper for its part is

6:27 doing its bit to enable customers like

6:30 welchire build that open ecosystem and I

6:33 would say nothing exemplifies this more

6:35 than the AI lab that Juniper you know

6:37 you know built in in our in our on our

6:40 premises where we have included

6:42 ecosystem vendors from GPU providers

6:45 like Nvidia and AMD we have Nick

6:48 providers like broadcom AMD and Nvidia

6:50 and we have uh storage Partners like VCA

6:53 and vast and in this entire ecosystem we

6:55 are actually testing these entire

6:57 ecosystem with real world AI workloads

6:59 both training as well as inference right

7:01 and and then eventually all of this is

7:03 Thoroughly tested and delivered as a

7:05 validated design which is a very simple

7:07 blueprint that customers can just take

7:09 and deploy for their cluster and be 100%

7:12 confident that the cluster is going to

7:14 operate uh at you know high efficience

7:16 and last but not least looking a little

7:17 bit more into the future um what should

7:20 potential clients expect how should can

7:22 they prepare to skill their AI

7:24 applications so look I think we seeing

7:27 that every year gpus and accelerators

7:29 are just getting more and more powerful

7:30 they're blasting more and more traffic

7:32 through the infrastructure through the

7:33 network uh we seeing models are just

7:35 getting larger and larger as well so

7:37 ethernet will have to keep up to those

7:39 challenges of scaling uh for multi-gpu

7:42 communication and much higher

7:43 performance in the future and the way we

7:45 are doing that is you know today

7:47 everybody is deploying based on 400 gig

7:48 and 800 gig ethernet but we are seeing

7:50 1.6 terabits per second coming right

7:52 around the Horizon in 2026 uh power

7:56 consumption is a very big issue in AI

7:58 clusters and Juniper for part is doing a

Importance of open ecosystems

8:00 lot to reduce power consumption for

8:02 Optics as well as it switches by almost

8:04 30 to 40% like that that's going to be

8:06 huge uh you know when we have that

8:08 coming in 2026 uh with things like

8:10 liquid cooling and everything else right

8:12 and then finally we are participating in

8:14 UE as well to further enable that open

8:17 ecosystem and you know to drive down you

8:19 know cost for our customers and on that

8:21 note of open I would maybe like to take

8:23 the opportunity to invite uh your uh

8:25 viewers to join our AI event AI Data

8:29 Center event on March 11th it's called

8:30 AI Unbound your data center your way and

8:34 in this event we're going to talk about

8:35 how Juniper uh broadcom and AMD are

8:39 enabling an open ecosystem for AI

8:41 infrastructure and really helping our

8:43 customers with validated designs to

8:45 accelerate that AI Journey yeah thanks

8:47 for the invite and um I would like to

8:49 thank everybody and of course join um as

8:52 well in March at the event you can

8:54 subscribe with a link in the in the

8:56 comment Raul thanks a lot great insight

8:59 and thank you for watching here for

9:00 Mobile World Congress at the unit the

9:02 booth and 20 2025 thank you for watching

9:06 and look forward to seeing you next time

9:11 [Music]

9:24 [Music]