[addrman] Change CAddrInfo access

Since knowledge of CAddrInfo is limited to callsites that import
addrman_impl.h, only objects in addrman.cpp or the tests have access. Thus we
can remove calling them friends and make the members public.

src/addrman_impl.h

@@ -17,7 +17,6 @@ public:
     //! last counted attempt (memory only)
     int64_t nLastCountAttempt{0};
 
-private:
     //! where knowledge about this address first came from
     CNetAddr source;
 
@@ -36,11 +35,6 @@ private:
     //! position in vRandom
     mutable int nRandomPos{-1};
 
-    friend class AddrManImpl;
-    friend class CAddrManDeterministic;
-
-public:
-
     SERIALIZE_METHODS(CAddrInfo, obj)
     {
         READWRITEAS(CAddress, obj);