Network models provide a unifying framework for understanding dependencies among variables in data-driven and engineering sciences. Networks can be used to reveal underlying data structures, infer functional modules, and facilitate experiment design. In practice, however, size, uncertainty and complexity of the underlying associations render these applications challenging. In this thesis, we illustrate the use of spectral, combinatorial, and statistical inference techniques in several network science problems. In Chapters 2-4, we consider network inference challenges. In Chapter 2, we introduce Network Maximal Correlation (NMC) as a multivariate measure of nonlinear association suitable for evaluation on large datasets. We characterize a solution of the NMC optimization using geometric properties of Hilbert spaces for finite discrete and jointly Gaussian random variables. We illustrate an application of NMC and multiple MC in inference of graphical models for bijective, possibly non-monotone, functions of jointly Gaussian variables. As a demonstration of NMC's utility, we infer nonlinear gene association networks and modules in cancer datasets and validate them using survival times of patients. In Chapter 3, we develop a network integration framework to infer gene regulatory networks in human and model organisms fly and worm using diverse and high-throughput datasets. Inferred regulatory interactions have significant overlap with known edges, indicating the robustness and accuracy of the proposed network inference framework. In Chapter 4, we formulate the transitive noise problem in networks as the inverse of matrix transitive closure and introduce an algorithm to solve it efficiently. We demonstrate the effectiveness of our approach in several applications such as regulatory network inference, protein contact map inference and strong collaboration tie inference. In Chapters 5-8, we consider network analysis challenges. In Chapter 5, we consider the problem of network alignment where the goal is to find a bijective mapping between nodes of two networks to maximize their overlapping edges while minimizing mismatches. This problem is essential in comparative analysis across large datasets and networks. To solve this combinatorial problem, we present a new scalable spectral algorithm which creates an eigenvector relaxation for the underlying optimization. We prove the optimality of the method under certain technical conditions, and show its effectiveness over various synthetic networks as well as in comparative analysis of gene regulatory networks across human, fly and worm species. In Chapter 6, we consider the source inference problem where the goal is to identify the source(s) of propagated signals across biological, social and engineered networks. To solve this problem, we propose a computationally tractable general method based on a path-based network diffusion kernel. We prove mean-field optimality of this method for different scenarios and show its effectiveness over several synthetic networks as well as in identifying sources in a Digg social news network. In Chapter 7, we consider the problem of learning low dimensional structures (such as clusters) in large networks. Here we introduce logistic Random Dot Product Graphs (RDPGs) as a new class of networks which includes most stochastic block models as well as other low dimensional structures. Using this model, we propose a scalable spectral method that solves the maximum likelihood inference problem asymptotically exactly. This leads to a new scalable spectral network clustering algorithm that is robust under different clustering setups. In Chapter 8, we consider the biclustering problem, the analog of clustering on bipartite graphs. This problem has several applications such as inference of co-regulated genes, document classification, and so on. Here we propose an algorithm based on message-passing that closely approximates a general likelihood function and excels at resolving the overlaps between biclusters. In Chapters 9-12, we consider design challenges of systems and algorithms for engineering networks such as communication networks. In Chapters 9-10, we create a connection between compressive sensing and traditional information theoretic techniques in source, channel and network coding and propose a joint coding scheme over wireless networks based on random projection and restricted eigenvalue principles. Moreover, we characterize fundamental results on the trade-off between the communication rate and the decoding complexity. In Chapters 11-12, we propose an adaptive nonuniform sampling framework, in which time increments between samples are determined as a function of the most recent increments and sample values, obviating the need to track time stamps. We analyze the performance of the proposed method for different stochastic and deterministic signal models and show its effectiveness to enhance measurements of heart ECG signals.