[bdNOG] BGP configuration issue in Caching Solution System

Aniruddha Barua aniruddha.barua at colbd.com
Sat Nov 23 16:59:02 BDT 2013


The traceroutes show that your caching server is acting like a router rather than a cache engine.
Either the redirection mechanism is not working as desired or the Caching server is not working as
expected. I wonder if Internet bound ICMP ping/tracroute packets should go to the Cache server at all.

Your problem is not just a routing issue, there is more. Whoever is providing you the BGP based
Caching Solution, they are the ones who can best help you in this regard.

All the best!

ANIRUDDHA BARUA
Email: aniruddha.barua at colbd.com, cto at colbd.com


---------- Original Message -----------
From: Mirza Md A Hu Al Rakib <mrakib at banglalinkgsm.com>
To: Aniruddha Barua <aniruddha.barua at colbd.com>, Mirza Rakib <mmrakib at gmail.com>, "nog at bdnog.org"
<nog at bdnog.org>
Sent: Sat, 23 Nov 2013 08:38:13 +0000
Subject: RE: [bdNOG] BGP configuration issue in Caching Solution System

> Hi Team,
> 
> No hope till now. Could you check the snap shot and suggest me what is the issue? Right now the
architecture is like:
> 
>        1. User side interface of Internet Router-116.58.207.249
>        2. eBGP with Cache Control Server: Internet Router Side-116.58.207.46/28, Cache Control
Server one interface-116.58.207.35/28
>        3. another interface of Cache Control Server-172.31.101.2/30 and internet router
side-172.31.101.1/30
> 
> We are getting loop. Traffic not going through BGP peer it goes other interface of the Cache
Control Server(Where BGP running for cache solution, it has two interface, one is with our Internet
Router and did the eBGP and another is just interfacing with default route from server).
> 
> Traffic should goes like:
> User---------->Internet Router-------->eBGP peer interface/Cache Control Server/Local Interface
> 
> Now going:
> 
> User---------->Internet Router--------> Local Interface /Cache Control Server/ ---------->
Internet Router------> Local Interface /Cache Control Server   : looping
> 
> Regards,
> Mirza Rakib
> 
> -----Original Message-----
> From: Mirza Md A Hu Al Rakib 
> Sent: Friday, November 22, 2013 2:26 PM
> To: 'Aniruddha Barua'; Mirza Rakib; nog at bdnog.org
> Subject: RE: [bdNOG] BGP configuration issue in Caching Solution System
> 
> Dear Barua Da,
> 
> As per our vendor allot: there is link between our Internet Router and Cache Application Server,
it mirror uplink traffic. Based on the uplink traffic one Caching Application Server where BGP
running, it update the BGP table from the mirror traffic and enforced the user request to redirect
the Cache server rather than internet.
> 
> I have planned to create sub interface with dot1Q. I think it will be solved the issue then.
> 
> Regards,
> Mirza Rakib
> -----Original Message-----
> From: Aniruddha Barua [mailto:aniruddha.barua at colbd.com]
> Sent: Friday, November 22, 2013 1:50 PM
> To: Aniruddha Barua; Mirza Md A Hu Al Rakib; Mirza Rakib; nog at bdnog.org
> Subject: Re: [bdNOG] BGP configuration issue in Caching Solution System
> 
> Dear Rakib bhai,
> 
> These servers cache web content. You said Cache BGP Server, so I was thinking they cache IP
Prefixes learned from BGP. I overlooked how they are supposed to work. So, need to know more.
> 
> Where is redirection to Caching servers taking place? In InternetRT or elsewhere? What is the
mechanism of redirection? I mean, should the Destination IPs in request packets get replaced with
Caching server IPs? If 'yes', it is not happening and that's why you are getting routing loop when
there is default route in Switch. If 'no', you'll need to put the servers and the router in same
LAN/VLAN. Discuss with your vendor about the mechanism first.
> 
> If OSPF is not running in your Cache servers, then the BGP routes they are getting from InternetRT
have 116.58.207.57 as GW but the servers' Default GW is 116.58.207.33 (VLAN10 IP of the Switch, I
guess).
> 
> Regards,
> 
> ANIRUDDHA BARUA
> Email: aniruddha.barua at colbd.com, cto at colbd.com
> 
> ---------- Original Message -----------
> From: "Aniruddha Barua" <aniruddha.barua at colbd.com>
> To: Mirza Md A Hu Al Rakib <mrakib at banglalinkgsm.com>, Mirza Rakib<mmrakib at gmail.com>,
"nog at bdnog.org" <nog at bdnog.org>
> Sent: Fri, 22 Nov 2013 12:09:15 +0600
> Subject: Re: [bdNOG] BGP configuration issue in Caching Solution System
> 
> > Dear Rakib bhai,
> > 
> > Of course there will be such problems if default route is there in the switch in this topology.
> > Actually you need a direct Layer 3 link between InternetRT and the 
> > Cache servers, i.e. they must belong to a single IP subnet. The L3 
> > switch is becoming a Layer 3 hop between the Router and the Cache servers. Your routing is fine,
but the packet flow logic in this setup is flawed.
> > 
> > I'm afraid, you'll have to rethink and redesign your topology. I 
> > suggest you go for standard dot1q trunking between Switch and 
> > InternetRT, carry all VLANs to the router. This way, the router and the Caching servers will be
in one VLAN (VLAN10) and there will be no such routing loop issue.
> > 
> > Regards,
> > 
> > ANIRUDDHA BARUA
> > Email: aniruddha.barua at colbd.com, cto at colbd.com
> > 
> > ---------- Original Message -----------
> > From: Mirza Md A Hu Al Rakib <mrakib at banglalinkgsm.com>
> > To: Aniruddha Barua <aniruddha.barua at colbd.com>, Mirza Rakib<mmrakib at gmail.com>, "nog at bdnog.org"
> > <nog at bdnog.org>
> > Sent: Fri, 22 Nov 2013 05:28:19 +0000
> > Subject: RE: [bdNOG] BGP configuration issue in Caching Solution 
> > System
> > 
> > > Dear Barua Da,
> > > 
> > > We have set the default route at Switch but it was an occurring an 
> > > issue. When user traffics are
> > trying redirect to Cache BGP server, in Switch it returning to 
> > Internet Router. As a result, user traffic not reached at Cache BGP 
> > Server.  Cache BGP Server do not have OSPF, but the subnet in OSPF network.
> > > 
> > > InternetRT #sh ip ospf route
> > > 
> > >             OSPF Router with ID (116.58.207.2) (Process ID 45245)
> > > 
> > >                 Base Topology (MTID 0)
> > > 
> > >     Area BACKBONE(0)
> > > 
> > >     Intra-area Route List
> > > *   116.58.207.56/30, Intra, cost 1, area 0, Connected
> > >       via 116.58.207.57, Port-channel3
> > > *   116.58.207.240/30, Intra, cost 1, area 0, Connected
> > >       via 116.58.207.241, Port-channel1
> > > *   116.58.207.248/30, Intra, cost 1, area 0, Connected
> > >       via 116.58.207.249, GigabitEthernet1/46
> > > *   116.58.207.244/30, Intra, cost 1, area 0, Connected
> > >       via 116.58.207.245, GigabitEthernet1/37
> > > *   116.58.207.252/30, Intra, cost 1, area 0, Connected
> > >       via 116.58.207.253, GigabitEthernet3/13 *>  116.58.207.228/30, 
> > > Intra, cost 2, area 0
> > >       via 116.58.207.254, GigabitEthernet3/13 *>  116.58.207.224/30, 
> > > Intra, cost 2, area 0
> > >       via 116.58.207.242, Port-channel1 *>  116.58.207.236/30, 
> > > Intra, cost 2, area 0
> > >       via 116.58.207.254, GigabitEthernet3/13
> > >       via 116.58.207.250, GigabitEthernet1/46 *>  116.58.207.32/28, 
> > > Intra, cost 2, area 0
> > >       via 116.58.207.58, Port-channel3 *>  116.58.207.48/30, Intra, 
> > > cost 2, area 0
> > >       via 116.58.207.58, Port-channel3 *>  116.58.207.52/30, Intra, 
> > > cost 2, area 0
> > >       via 116.58.207.58, Port-channel3 *>  116.58.207.1/32, Intra, 
> > > cost 2, area 0
> > >       via 116.58.207.254, GigabitEthernet3/13
> > > *   116.58.207.2/32, Intra, cost 1, area 0, Connected
> > >       via 116.58.207.2, Loopback0
> > > *>  116.58.207.3/32, Intra, cost 2, area 0
> > >       via 116.58.207.246, GigabitEthernet1/37 *>  116.58.207.4/32, 
> > > Intra, cost 2, area 0
> > >       via 116.58.207.250, GigabitEthernet1/46 *>  116.58.207.5/32, 
> > > Intra, cost 2, area 0
> > >       via 116.58.207.242, Port-channel1 *>  116.58.207.6/32, Intra, 
> > > cost 3, area 0
> > >       via 116.58.207.254, GigabitEthernet3/13
> > >       via 116.58.207.242, Port-channel1
> > > 
> > > -----Original Message-----
> > > From: Aniruddha Barua [mailto:aniruddha.barua at colbd.com]
> > > Sent: Friday, November 22, 2013 11:20 AM
> > > To: Mirza Md A Hu Al Rakib; Mirza Rakib; nog at bdnog.org
> > > Subject: RE: [bdNOG] BGP configuration issue in Caching Solution 
> > > System
> > > 
> > > Dear Rakib bhai,
> > > 
> > > What is the default route in the Switch? I see it is not set. Set it 
> > > manually or get it from
> > InternetRT over OSPF.
> > > 
> > > Please also show us the output of "show ip route ospf" from 
> > > InternetRT and the same from
> CACHE-SERVER.
> > > 
> > > ANIRUDDHA BARUA
> > > Email: aniruddha.barua at colbd.com, cto at colbd.com
> > > 
> > > ---------- Original Message -----------
> > > From: Mirza Md A Hu Al Rakib <mrakib at banglalinkgsm.com>
> > > To: Aniruddha Barua <aniruddha.barua at colbd.com>, Mirza Rakib<mmrakib at gmail.com>, "nog at bdnog.org"
> > > <nog at bdnog.org>
> > > Sent: Fri, 22 Nov 2013 05:02:47 +0000
> > > Subject: RE: [bdNOG] BGP configuration issue in Caching Solution 
> > > System
> > > 
> > > > Dear Barua Da,
> > > > 
> > > > We are using OSPF in L3 and InternetRT. Server block1(BGP Server) 
> > > > use default route to VLAN10,
> > > Server block2(Cache Server) use default route to VLAN20 and server 
> > > block3(Cache Server) use
> > default route to VLAN30 of L3 Switch.
> > > > 
> > > > We are using "next-hop-self" in InternetRT. The server is 
> > > > customized MSWE-Cache by allot, I see
> > > it's Linux kernel.
> > > > 
> > > > MSWE-CACHE-SW#sh ip route
> > > > 
> > > > Gateway of last resort is not set
> > > > 
> > > >      116.0.0.0/8 is variably subnetted, 17 subnets, 3 masks
> > > > O       116.58.207.224/30 [110/3] via 116.58.207.57, 00:23:24, Port-channel1
> > > > O       116.58.207.228/30 [110/3] via 116.58.207.57, 00:23:24, Port-channel1
> > > > O       116.58.207.236/30 [110/3] via 116.58.207.57, 00:23:24, Port-channel1
> > > > O       116.58.207.240/30 [110/2] via 116.58.207.57, 00:23:24, Port-channel1
> > > > O       116.58.207.244/30 [110/2] via 116.58.207.57, 00:23:24, Port-channel1
> > > > O       116.58.207.248/30 [110/2] via 116.58.207.57, 00:23:24, Port-channel1
> > > > O       116.58.207.252/30 [110/2] via 116.58.207.57, 00:23:24, Port-channel1
> > > > O       116.58.207.1/32 [110/3] via 116.58.207.57, 00:23:24, Port-channel1
> > > > O       116.58.207.3/32 [110/3] via 116.58.207.57, 00:23:24, Port-channel1
> > > > O       116.58.207.2/32 [110/2] via 116.58.207.57, 00:23:24, Port-channel1
> > > > O       116.58.207.5/32 [110/3] via 116.58.207.57, 00:23:24, Port-channel1
> > > > O       116.58.207.4/32 [110/3] via 116.58.207.57, 00:23:24, Port-channel1
> > > > O       116.58.207.6/32 [110/4] via 116.58.207.57, 00:23:24, Port-channel1
> > > > C       116.58.207.32/28 is directly connected, Vlan10
> > > > C       116.58.207.48/30 is directly connected, Vlan20
> > > > C       116.58.207.52/30 is directly connected, Vlan30
> > > > C       116.58.207.56/30 is directly connected, Port-channel1
> > > > 
> > > > =========================================================
> > > > InternetRT #sh ip route
> > > > Gateway of last resort is 114.130.3.141 to network 0.0.0.0
> > > > 
> > > > B*    0.0.0.0/0 [20/0] via 114.130.3.141, 2w0d
> > > >       1.0.0.0/8 is variably subnetted, 1555 subnets, 14 masks
> > > > B        1.0.0.0/24 [20/0] via 114.130.3.141, 1w1d
> > > > B        1.0.4.0/24 [20/0] via 114.130.3.141, 2w0d
> > > > B        1.0.5.0/24 [20/0] via 114.130.3.141, 2w0d
> > > > B- - -- - - - - - -
> > > > -------------------
> > > > 
> > > > CACHE-SERVER#
> > > > (managed by our vendor Allot, I will provide you the show route today evening).
> > > > 
> > > > Regards,
> > > > Mirza Rakib
> > > > -----Original Message-----
> > > > From: nog-bounces at bdnog.org [mailto:nog-bounces at bdnog.org] On 
> > > > Behalf Of Aniruddha Barua
> > > > Sent: Friday, November 22, 2013 10:19 AM
> > > > To: Mirza Rakib; nog at bdnog.org
> > > > Subject: Re: [bdNOG] BGP configuration issue in Caching Solution 
> > > > System
> > > > 
> > > > Dear Mr. Rakib,
> > > > 
> > > > In this topology, your L3 switch has become a router and I presume 
> > > > you have static routing in
> > > InternetRT, Servers and L3 Switch to find each other. Instead, 
> > > enable OSPF (or RIP) in L3 Switch,
> > InternetRT and the Caching Servers if supported by all.
> > > > 
> > > > In present scenario, your Caching servers are getting Internet 
> > > > routes from InternetRT with next
> > > hop address 116.58.207.57, if next-hop-self is configured in 
> > > InternetRT. 116.58.207.57 is on
> > different LAN segment from the servers. What is the OS of Caching 
> > servers? Do the Caching servers support recursive route lookup? If 
> > these are Linux based servers, you need to find a way (route filters 
> > may
> > > > be) to modify the Target Scope of all routes received from 
> > > > InternetRT over iBGP. Better, enable
> > > OSPF (or RIP) as I mentioned earlier.
> > > > 
> > > > Regards,
> > > > 
> > > > ANIRUDDHA BARUA
> > > >  Email: aniruddha.barua at colbd.com, cto at colbd.com
> > > > 
> > > > ---------- Original Message -----------
> > > >  From: Mirza Rakib <mmrakib at gmail.com>
> > > >  To: nog at bdnog.org
> > > >  Sent: Thu, 21 Nov 2013 22:23:59 +0600
> > > >  Subject: Re: [bdNOG] BGP configuration issue in Caching Solution 
> > > > System
> > > > 
> > > > > Hi Team,
> > > > > 
> > > > > Please have the architecture for your better understanding: 
> > > > > 
> > > > >   
> > > > > 
> > > > > On Thu, Nov 21, 2013 at 9:41 PM, Mirza Rakib <mmrakib at gmail.com> wrote: 
> > > > >  
> > > > > 
> > > > > Hi Team,
> > > > >   
> > > > > I am facing an issue to configure internal BGP (iBGP) where a L3 
> > > > > switch exist in between Router
> > > > and another Caching Solution Server. The architecture like: 
> > > > >   
> > > > > Router-A(BGP, AS45245)----L3 Switch(3 different VALNs: VLAN10,
> > > > > VLAN20 and VLAN30)-------BGP
> > > > Caching Control Server( AS45245, VLAN10)
> > > > >   
> > > > >   
> > > > > BGP formed but there is routing issue. We can reach Router-A to 
> > > > > all Servers in differnet VLANs and
> > > >  vice versa but cannot reach outside and also from outside to 
> > > > servers are not reachable. We are
> > > getting some sites IP in BGP Caching Control Server bgp table. 
> > > However we not reacing there, we got
> > > > loop or stack at L3 switch, whenever we trace.   
> > > > >   
> > > > > Sincerely,
> > > > > Mirza Rakib
> > > > > 
> > > > >  
> > > > >  
> > > > > --
> > > > > Sincerely,
> > > > > Mirza Rakib
> > > > > SUST CSE-2K1
> > > > ------- End of Original Message -------
> > > > 
> > > > _______________________________________________
> > > > nog mailing list
> > > > nog at bdnog.org
> > > > http://mailman.bdnog.org/mailman/listinfo/nog
> > > > 
> > > > This email and any files transmitted with it are confidential and 
> > > > intended solely for the use of
> > > the individual or entity to whom it is addressed. It may also be 
> > > legally privileged. If you are
> > not the named addressee, you should not copy, forward, disclose or use 
> > any part of it. If you have received this message in error, please 
> > delete it and all copies from your system and notify the sender 
> > immediately by return e-mail. Internet communications cannot be guaranteed to be timely, secure,
error or virus-free. The sender does not accept liability for any errors or omissions.
> > > ------- End of Original Message -------
> > ------- End of Original Message -------
> > 
> > _______________________________________________
> > nog mailing list
> > nog at bdnog.org
> > http://mailman.bdnog.org/mailman/listinfo/nog
> ------- End of Original Message -------
------- End of Original Message -------



More information about the nog mailing list